2024-04-25T20:39:05.782 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T20:39:05.783 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_leonidus@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --archive /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652 --name leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi --verbose -- /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/orig.config.yaml 2024-04-25T20:39:05.839 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T20:39:05.932 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652 branch: wip-lusov-quiescer description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: null first_in_suite: false job_id: '7673652' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - nowsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: e221b808d285b9d02e54cc10c53c84188e8e41e3 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2919 sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph suite_sha1: e221b808d285b9d02e54cc10c53c84188e8e41e3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIcMo5SVjzeCCC9sX+g6yCA3BHi6bGDMpaeg4N3zUpJNBk/2sx1ywEcH0R/R6D/f+T/VpBpD87A1Sk+bVp43+h0= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI10lUouKjO775cv66l7qvkSNBu26z/d1nD6syQgF66fa5DXV/dvpIqnAE1ec6W62FPkrM3dkp2j2LhOAYNI9uk= smithi139.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQ507FFNV3QplQfR3lBTH6lwv6uEKPVXzfIlGTtaFkZBYIEzDHzi1y+Il8x28vMzI9kkN+N5K+4HkmkLXLIGAE= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_19:59:35 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T20:39:05.932 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa; will attempt to use it 2024-04-25T20:39:05.933 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks 2024-04-25T20:39:05.933 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T20:39:05.934 INFO:teuthology.task.internal:Checking packages... 2024-04-25T20:39:05.955 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd69f13d827952bb07bb9b7973bca14f97d764d6f' 2024-04-25T20:39:05.955 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T20:39:05.955 INFO:teuthology.packaging:ref: None 2024-04-25T20:39:05.955 INFO:teuthology.packaging:tag: None 2024-04-25T20:39:05.955 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-25T20:39:05.955 INFO:teuthology.packaging:sha1: d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:39:05.955 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-25T20:39:06.204 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3156.gd69f13d8 2024-04-25T20:39:06.204 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T20:39:06.214 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T20:39:06.214 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T20:39:06.241 INFO:teuthology.task.internal:Saving configuration 2024-04-25T20:39:06.261 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T20:39:06.267 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T20:39:06.293 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:34:11.728484', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIcMo5SVjzeCCC9sX+g6yCA3BHi6bGDMpaeg4N3zUpJNBk/2sx1ywEcH0R/R6D/f+T/VpBpD87A1Sk+bVp43+h0='} 2024-04-25T20:39:06.312 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:34:11.730323', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI10lUouKjO775cv66l7qvkSNBu26z/d1nD6syQgF66fa5DXV/dvpIqnAE1ec6W62FPkrM3dkp2j2LhOAYNI9uk='} 2024-04-25T20:39:06.330 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:34:11.731662', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQ507FFNV3QplQfR3lBTH6lwv6uEKPVXzfIlGTtaFkZBYIEzDHzi1y+Il8x28vMzI9kkN+N5K+4HkmkLXLIGAE='} 2024-04-25T20:39:06.330 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T20:39:06.337 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-25T20:39:06.337 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-25T20:39:06.337 INFO:teuthology.task.internal:roles: ubuntu@smithi139.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-25T20:39:06.337 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T20:39:06.436 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5cd5dae0d0>, signals=[15]) 2024-04-25T20:39:06.436 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T20:39:06.443 INFO:teuthology.task.internal:Opening connections... 2024-04-25T20:39:06.443 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:39:06.445 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:39:06.519 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T20:39:06.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:39:06.591 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T20:39:06.592 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:39:06.663 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T20:39:06.670 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-04-25T20:39:06.688 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-04-25T20:39:06.688 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-04-25T20:39:06.745 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-04-25T20:39:06.745 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-04-25T20:39:06.745 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-04-25T20:39:06.745 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:39:06.746 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:39:06.747 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-04-25T20:39:06.768 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2024-04-25T20:39:06.786 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2024-04-25T20:39:06.786 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2024-04-25T20:39:06.842 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="CentOS Stream" 2024-04-25T20:39:06.842 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="9" 2024-04-25T20:39:06.842 INFO:teuthology.orchestra.run.smithi097.stdout:ID="centos" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="9" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:39:06.843 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:39:06.843 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2024-04-25T20:39:06.868 DEBUG:teuthology.orchestra.run.smithi139:> uname -m 2024-04-25T20:39:06.886 INFO:teuthology.orchestra.run.smithi139.stdout:x86_64 2024-04-25T20:39:06.886 DEBUG:teuthology.orchestra.run.smithi139:> cat /etc/os-release 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:NAME="CentOS Stream" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION="9" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:ID="centos" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:ID_LIKE="rhel fedora" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:VERSION_ID="9" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:PLATFORM_ID="platform:el9" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:ANSI_COLOR="0;31" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:LOGO="fedora-logo-icon" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:HOME_URL="https://centos.org/" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T20:39:06.943 INFO:teuthology.orchestra.run.smithi139.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T20:39:06.944 INFO:teuthology.lock.ops:Updating smithi139.front.sepia.ceph.com on lock server 2024-04-25T20:39:06.968 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T20:39:06.978 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T20:39:06.983 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T20:39:06.983 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:39:06.986 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:39:06.988 DEBUG:teuthology.orchestra.run.smithi139:> test '!' -e /home/ubuntu/cephtest 2024-04-25T20:39:07.004 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T20:39:07.009 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T20:39:07.010 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:39:07.043 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:39:07.045 DEBUG:teuthology.orchestra.run.smithi139:> test -z $(ls -A /var/lib/ceph) 2024-04-25T20:39:07.129 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T20:39:07.208 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T20:39:07.222 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T20:39:07.222 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-25T20:39:07.222 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-25T20:39:07.222 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:39:07.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:39:07.239 DEBUG:teuthology.orchestra.run.smithi096:> uname -r 2024-04-25T20:39:07.296 INFO:teuthology.orchestra.run.smithi096.stdout:5.14.0-437.el9.x86_64 2024-04-25T20:39:07.296 INFO:teuthology.task.kernel:Running kernel on smithi096: 5.14.0-437.el9.x86_64 2024-04-25T20:39:07.297 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum install -y kernel 2024-04-25T20:39:09.876 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-25T20:39:11.736 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T20:39:15.894 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-25T20:39:17.542 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T20:39:37.060 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T20:39:42.064 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-25T20:39:43.937 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T20:39:43.937 INFO:teuthology.orchestra.run.smithi096.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T20:39:44.037 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T20:39:44.038 INFO:teuthology.orchestra.run.smithi096.stdout:Nothing to do. 2024-04-25T20:39:44.038 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T20:39:44.142 DEBUG:teuthology.orchestra.run.smithi096:> echo no | sudo yum reinstall kernel || true 2024-04-25T20:39:44.644 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 08:39:42 PM UTC. 2024-04-25T20:39:44.766 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 255 k 2024-04-25T20:39:44.776 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-04-25T20:39:44.777 INFO:teuthology.orchestra.run.smithi096.stderr:Operation aborted. 2024-04-25T20:39:44.839 INFO:teuthology.orchestra.run.smithi096.stdout:Is this ok [y/N]: 2024-04-25T20:39:44.839 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum reinstall -y kernel || true 2024-04-25T20:39:45.327 INFO:teuthology.orchestra.run.smithi096.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:39:42 PM UTC. 2024-04-25T20:39:45.443 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: Package Architecture Version Repository Size 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalling: 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout:================================================================================ 2024-04-25T20:39:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:39:45.445 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 255 k 2024-04-25T20:39:45.445 INFO:teuthology.orchestra.run.smithi096.stdout:Installed size: 0 2024-04-25T20:39:45.445 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-25T20:39:46.207 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-437.el9.x86_64.rpm 509 kB/s | 255 kB 00:00 2024-04-25T20:39:46.208 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:39:46.208 INFO:teuthology.orchestra.run.smithi096.stdout:Total 334 kB/s | 255 kB 00:00 2024-04-25T20:39:46.229 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-25T20:39:46.413 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-25T20:39:46.413 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-25T20:39:46.418 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-25T20:39:46.418 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-25T20:39:46.562 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-25T20:39:46.633 INFO:teuthology.orchestra.run.smithi096.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:39:46.634 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:39:47.283 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:39:47.283 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T20:39:47.970 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T20:39:47.970 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:39:47.970 INFO:teuthology.orchestra.run.smithi096.stdout:Reinstalled: 2024-04-25T20:39:47.971 INFO:teuthology.orchestra.run.smithi096.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T20:39:47.971 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:39:47.971 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T20:39:48.143 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T20:39:48.191 INFO:teuthology.orchestra.run.smithi096.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T20:39:48.191 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T20:39:48.191 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T20:39:48.191 DEBUG:teuthology.parallel:result is None 2024-04-25T20:39:48.192 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T20:39:48.198 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T20:39:48.199 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:39:48.201 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:39:48.203 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T20:39:48.221 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T20:39:48.230 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T20:39:48.236 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T20:39:48.237 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:39:48.255 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:39:48.261 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T20:39:48.379 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T20:39:48.386 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T20:39:48.386 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:39:48.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:39:48.402 DEBUG:teuthology.orchestra.run.smithi139:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T20:39:48.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:39:48.418 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:39:48.421 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:39:48.445 DEBUG:teuthology.orchestra.run.smithi139:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T20:39:48.452 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:39:48.470 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:39:48.478 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:39:48.511 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:39:48.512 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:39:48.528 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T20:39:48.530 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T20:39:48.537 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T20:39:48.537 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:39:48.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:39:48.556 DEBUG:teuthology.orchestra.run.smithi139:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T20:39:48.585 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T20:39:48.593 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T20:39:48.593 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:39:48.602 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:39:48.623 DEBUG:teuthology.orchestra.run.smithi139:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T20:39:48.641 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:39:48.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:39:48.747 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:39:48.832 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:39:48.866 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:39:48.866 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:39:48.930 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:39:48.973 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:39:49.033 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:39:49.118 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:39:49.160 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:39:49.161 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:39:49.226 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:39:49.272 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T20:39:49.314 DEBUG:teuthology.orchestra.run.smithi139:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:39:49.396 DEBUG:teuthology.orchestra.run.smithi139:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T20:39:49.438 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:39:49.438 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T20:39:49.504 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-04-25T20:39:49.506 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2024-04-25T20:39:49.508 DEBUG:teuthology.orchestra.run.smithi139:> sudo service rsyslog restart 2024-04-25T20:39:49.542 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:39:49.544 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:39:49.590 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T20:39:50.037 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T20:39:50.044 INFO:teuthology.task.internal:Starting timer... 2024-04-25T20:39:50.045 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T20:39:50.072 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T20:39:50.079 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-25T20:39:50.079 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-04-25T20:39:50.140 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-04-25T20:39:50.141 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2024-04-25T20:39:50.201 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2024-04-25T20:39:50.203 DEBUG:teuthology.orchestra.run.smithi139:> sudo service auditd rotate 2024-04-25T20:39:50.277 INFO:teuthology.orchestra.run.smithi139.stdout:Rotating logs: 2024-04-25T20:39:50.279 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T20:39:50.279 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-04-25T20:39:50.298 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-04-25T20:39:50.299 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2024-04-25T20:39:50.326 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2024-04-25T20:39:50.326 DEBUG:teuthology.orchestra.run.smithi139:> /usr/sbin/getenforce 2024-04-25T20:39:50.356 INFO:teuthology.orchestra.run.smithi139.stdout:Permissive 2024-04-25T20:39:50.357 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi139.front.sepia.ceph.com': 'permissive'} 2024-04-25T20:39:50.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:39:50.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:39:50.383 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:39:50.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:39:50.409 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T20:39:50.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:39:50.435 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T20:39:50.435 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:39:50.460 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:39:50.486 DEBUG:teuthology.orchestra.run.smithi139:> sudo /usr/sbin/setenforce permissive 2024-04-25T20:39:50.512 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T20:39:50.522 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-25T20:39:50.527 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-25T20:39:50.650 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T20:39:50.665 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T20:39:50.667 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi096.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi139.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T20:43:34.543 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi139.front.sepia.ceph.com')] 2024-04-25T20:43:34.544 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-25T20:43:34.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:43:34.620 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-04-25T20:43:34.697 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-04-25T20:43:34.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T20:43:34.698 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:43:34.772 DEBUG:teuthology.orchestra.run.smithi097:> true 2024-04-25T20:43:34.846 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2024-04-25T20:43:34.846 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-04-25T20:43:34.847 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi139.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T20:43:34.920 DEBUG:teuthology.orchestra.run.smithi139:> true 2024-04-25T20:43:34.998 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi139.front.sepia.ceph.com' 2024-04-25T20:43:34.998 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T20:43:35.010 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T20:43:35.010 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:43:35.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:43:35.013 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:43:35.013 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:43:35.015 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T20:43:35.015 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T20:43:35.040 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:43:35.042 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:43:35.053 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:43:35.056 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:43:35.067 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T20:43:35.077 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-04-25T20:43:35.080 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2024-04-25T20:43:35.080 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T20:43:35.087 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-04-25T20:43:35.090 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2024-04-25T20:43:35.100 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:43:35.103 INFO:teuthology.orchestra.run.smithi139.stderr:sudo: ntpd: command not found 2024-04-25T20:43:35.104 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:43:35.112 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:43:35.114 INFO:teuthology.orchestra.run.smithi139.stdout:506 Cannot talk to daemon 2024-04-25T20:43:35.117 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:43:35.127 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T20:43:35.139 INFO:teuthology.orchestra.run.smithi139.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T20:43:35.165 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:43:35.169 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:43:35.169 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-25T20:43:35.169 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.169 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.169 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.170 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.176 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:43:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:43:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-25T20:43:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.180 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.194 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-04-25T20:43:35.198 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T20:43:35.198 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-04-25T20:43:35.199 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.199 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.199 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.199 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T20:43:35.199 INFO:teuthology.run_tasks:Running task install... 2024-04-25T20:43:35.207 DEBUG:teuthology.task.install:project ceph 2024-04-25T20:43:35.207 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'}} 2024-04-25T20:43:35.208 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f'} 2024-04-25T20:43:35.208 INFO:teuthology.task.install:Using flavor: default 2024-04-25T20:43:35.213 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T20:43:35.213 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T20:43:35.214 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:43:35.214 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:43:35.216 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:43:35.217 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:43:35.218 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'tag': None, 'wait_for_package': False} 2024-04-25T20:43:35.218 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:43:35.368 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:43:35.369 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:43:35.438 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:43:35.438 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:43:35.455 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/ 2024-04-25T20:43:35.455 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3156.gd69f13d8 2024-04-25T20:43:35.488 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:43:35.488 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:43:35.488 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:43:35.513 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:43:35.513 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:43:35.555 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:43:35.555 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:43:35.556 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:43:35.570 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T20:43:35.570 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:43:35.570 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T20:43:35.577 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:43:35.581 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:43:35.582 DEBUG:teuthology.orchestra.run.smithi139:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:43:35.595 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T20:43:35.595 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d69f13d827952bb07bb9b7973bca14f97d764d6f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T20:43:35.646 DEBUG:teuthology.orchestra.run.smithi139:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:43:35.649 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:43:35.660 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T20:43:35.712 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2024-04-25T20:43:35.714 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2024-04-25T20:43:35.718 DEBUG:teuthology.orchestra.run.smithi139:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:43:35.733 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T20:43:35.781 INFO:teuthology.orchestra.run.smithi139.stdout:check_obsoletes = 1 2024-04-25T20:43:35.783 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum clean all 2024-04-25T20:43:35.796 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-04-25T20:43:35.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-04-25T20:43:35.937 INFO:teuthology.orchestra.run.smithi097.stdout:48 files removed 2024-04-25T20:43:35.964 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:43:36.006 INFO:teuthology.orchestra.run.smithi139.stdout:48 files removed 2024-04-25T20:43:36.022 INFO:teuthology.orchestra.run.smithi096.stdout:48 files removed 2024-04-25T20:43:36.033 DEBUG:teuthology.orchestra.run.smithi139:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:43:36.049 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T20:43:36.644 INFO:teuthology.orchestra.run.smithi139.stdout:ceph packages for x86_64 236 kB/s | 83 kB 00:00 2024-04-25T20:43:36.683 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 223 kB/s | 83 kB 00:00 2024-04-25T20:43:36.694 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-04-25T20:43:37.042 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-25T20:43:37.069 INFO:teuthology.orchestra.run.smithi139.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T20:43:37.128 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-04-25T20:43:37.369 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-04-25T20:43:37.462 INFO:teuthology.orchestra.run.smithi139.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-25T20:43:37.470 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-04-25T20:43:38.006 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-25T20:43:38.113 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.0 MB 00:00 2024-04-25T20:43:38.306 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - BaseOS 9.9 MB/s | 8.0 MB 00:00 2024-04-25T20:43:39.841 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-04-25T20:43:40.090 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 21 MB/s | 19 MB 00:00 2024-04-25T20:43:40.524 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-25T20:43:44.316 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.2 MB 00:00 2024-04-25T20:43:44.407 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 8.6 MB/s | 6.2 MB 00:00 2024-04-25T20:43:44.657 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-04-25T20:43:45.681 INFO:teuthology.orchestra.run.smithi097.stdout:CentOS Stream 9 - Extras packages 74 kB/s | 16 kB 00:00 2024-04-25T20:43:46.089 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-04-25T20:43:46.432 INFO:teuthology.orchestra.run.smithi139.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-25T20:43:47.723 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-04-25T20:43:48.149 INFO:teuthology.orchestra.run.smithi139.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-25T20:43:48.477 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-25T20:43:52.722 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-25T20:43:53.274 INFO:teuthology.orchestra.run.smithi139.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-04-25T20:43:53.541 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T20:43:54.865 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:43:54.865 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:43:54.876 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:43:54.876 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:43:54.877 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:43:54.878 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:43:54.880 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:43:54.882 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:43:54.883 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:43:54.943 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-04-25T20:43:54.952 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T20:43:54.952 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:43:54.953 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:43:54.954 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:43:54.955 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:43:54.956 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:43:54.957 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:43:54.958 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:43:54.959 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:43:54.960 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:43:54.961 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:43:54.962 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:43:54.963 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:43:54.964 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 4 Packages 2024-04-25T20:43:54.965 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:43:54.966 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 222 M 2024-04-25T20:43:54.966 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-04-25T20:43:55.399 INFO:teuthology.orchestra.run.smithi139.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:43:55.399 INFO:teuthology.orchestra.run.smithi139.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:43:55.410 INFO:teuthology.orchestra.run.smithi139.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:43:55.411 INFO:teuthology.orchestra.run.smithi139.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:43:55.412 INFO:teuthology.orchestra.run.smithi139.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:43:55.413 INFO:teuthology.orchestra.run.smithi139.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:43:55.414 INFO:teuthology.orchestra.run.smithi139.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:43:55.416 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:43:55.417 INFO:teuthology.orchestra.run.smithi139.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:43:55.477 INFO:teuthology.orchestra.run.smithi139.stdout:Dependencies resolved. 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout: Package Arch Version Repository Size 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout:Installing: 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:43:55.494 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:43:55.495 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:43:55.496 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:43:55.497 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrading: 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout:Installing dependencies: 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:43:55.498 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:43:55.499 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:43:55.500 INFO:teuthology.orchestra.run.smithi139.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:43:55.501 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:43:55.502 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:43:55.503 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:43:55.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:43:55.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:43:55.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:43:55.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:43:55.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:43:55.504 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:43:55.505 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:43:55.505 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:43:55.505 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:43:55.505 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:43:55.505 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:43:55.505 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:43:55.506 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:43:55.506 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:43:55.506 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:43:55.506 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:43:55.506 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:43:55.506 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:43:55.507 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:43:55.507 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:43:55.507 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:43:55.507 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:43:55.507 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:43:55.507 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:43:55.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:43:55.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:43:55.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:43:55.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:43:55.508 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:43:55.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:43:55.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:43:55.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:43:55.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:43:55.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:43:55.509 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:43:55.510 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:43:55.510 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:43:55.510 INFO:teuthology.orchestra.run.smithi139.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:43:55.510 INFO:teuthology.orchestra.run.smithi139.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:43:55.510 INFO:teuthology.orchestra.run.smithi139.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:43:55.510 INFO:teuthology.orchestra.run.smithi139.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:43:55.511 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:43:55.511 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:43:55.511 INFO:teuthology.orchestra.run.smithi139.stdout:Installing weak dependencies: 2024-04-25T20:43:55.511 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:43:55.511 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:43:55.512 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction Summary 2024-04-25T20:43:55.512 INFO:teuthology.orchestra.run.smithi139.stdout:======================================================================================= 2024-04-25T20:43:55.512 INFO:teuthology.orchestra.run.smithi139.stdout:Install 148 Packages 2024-04-25T20:43:55.512 INFO:teuthology.orchestra.run.smithi139.stdout:Upgrade 4 Packages 2024-04-25T20:43:55.512 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:43:55.513 INFO:teuthology.orchestra.run.smithi139.stdout:Total download size: 222 M 2024-04-25T20:43:55.513 INFO:teuthology.orchestra.run.smithi139.stdout:Downloading Packages: 2024-04-25T20:43:55.667 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:43:55.667 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T20:43:55.678 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T20:43:55.679 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T20:43:55.680 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T20:43:55.681 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T20:43:55.682 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T20:43:55.685 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T20:43:55.685 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T20:43:55.745 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2024-04-25T20:43:55.761 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T20:43:55.761 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 6.6 k 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 890 k 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 145 k 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 1.5 M 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 142 k 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 3.5 M 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 7.4 M 2024-04-25T20:43:55.762 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 49 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 11 M 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 49 M 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 273 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 762 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 225 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 27 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 18 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 32 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 747 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 126 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T20:43:55.763 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 161 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 320 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 298 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 99 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 85 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 170 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2024-04-25T20:43:55.764 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.4 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 3.1 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 21 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 34 k 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 2.4 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 249 k 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 4.7 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 17 M 2024-04-25T20:43:55.765 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3156.gd69f13d8.el7 ceph-noarch 16 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 25 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 163 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T20:43:55.766 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 500 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 5.4 M 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T20:43:55.767 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 45 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:19.0.0-3156.gd69f13d8.el7 ceph 135 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T20:43:55.768 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T20:43:55.769 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T20:43:55.770 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T20:43:55.771 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T20:43:55.772 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout:======================================================================================= 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout:Install 148 Packages 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 4 Packages 2024-04-25T20:43:55.773 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:43:55.774 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 222 M 2024-04-25T20:43:55.774 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2024-04-25T20:43:55.807 INFO:teuthology.orchestra.run.smithi096.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 38 kB/s | 6.6 kB 00:00 2024-04-25T20:43:56.090 INFO:teuthology.orchestra.run.smithi096.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 3.1 MB/s | 890 kB 00:00 2024-04-25T20:43:56.157 INFO:teuthology.orchestra.run.smithi096.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-25T20:43:56.166 INFO:teuthology.orchestra.run.smithi139.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-25T20:43:56.491 INFO:teuthology.orchestra.run.smithi139.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 2.7 MB/s | 890 kB 00:00 2024-04-25T20:43:56.549 INFO:teuthology.orchestra.run.smithi096.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 6.0 MB/s | 2.4 MB 00:00 2024-04-25T20:43:56.558 INFO:teuthology.orchestra.run.smithi139.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-25T20:43:56.577 INFO:teuthology.orchestra.run.smithi097.stdout:(1/152): ceph-19.0.0-3156.gd69f13d8.el7.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-25T20:43:56.866 INFO:teuthology.orchestra.run.smithi096.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 4.7 MB/s | 1.5 MB 00:00 2024-04-25T20:43:57.433 INFO:teuthology.orchestra.run.smithi139.stdout:(4/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 2.7 MB/s | 2.4 MB 00:00 2024-04-25T20:43:57.475 INFO:teuthology.orchestra.run.smithi096.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.9 MB/s | 5.4 MB 00:01 2024-04-25T20:43:57.519 INFO:teuthology.orchestra.run.smithi097.stdout:(2/152): ceph-fuse-19.0.0-3156.gd69f13d8.el7.x8 945 kB/s | 890 kB 00:00 2024-04-25T20:43:57.728 INFO:teuthology.orchestra.run.smithi097.stdout:(3/152): ceph-immutable-object-cache-19.0.0-315 695 kB/s | 145 kB 00:00 2024-04-25T20:43:57.908 INFO:teuthology.orchestra.run.smithi096.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 4.5 MB/s | 4.7 MB 00:01 2024-04-25T20:43:57.992 INFO:teuthology.orchestra.run.smithi139.stdout:(5/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.7 MB/s | 1.5 MB 00:00 2024-04-25T20:43:58.433 INFO:teuthology.orchestra.run.smithi096.stdout:(8/152): ceph-common-19.0.0-3156.gd69f13d8.el7. 7.5 MB/s | 21 MB 00:02 2024-04-25T20:43:58.459 INFO:teuthology.orchestra.run.smithi139.stdout:(6/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.2 MB/s | 5.4 MB 00:02 2024-04-25T20:43:58.478 INFO:teuthology.orchestra.run.smithi097.stdout:(4/152): ceph-base-19.0.0-3156.gd69f13d8.el7.x8 2.5 MB/s | 5.4 MB 00:02 2024-04-25T20:43:58.491 INFO:teuthology.orchestra.run.smithi096.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 432 kB/s | 25 kB 00:00 2024-04-25T20:43:58.945 INFO:teuthology.orchestra.run.smithi097.stdout:(5/152): ceph-mds-19.0.0-3156.gd69f13d8.el7.x86 1.9 MB/s | 2.4 MB 00:01 2024-04-25T20:43:59.090 INFO:teuthology.orchestra.run.smithi097.stdout:(6/152): ceph-mgr-19.0.0-3156.gd69f13d8.el7.x86 2.4 MB/s | 1.5 MB 00:00 2024-04-25T20:43:59.459 INFO:teuthology.orchestra.run.smithi139.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.2 MB/s | 4.7 MB 00:01 2024-04-25T20:44:00.242 INFO:teuthology.orchestra.run.smithi096.stdout:(10/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el 4.5 MB/s | 11 MB 00:02 2024-04-25T20:44:00.320 INFO:teuthology.orchestra.run.smithi097.stdout:(7/152): ceph-mon-19.0.0-3156.gd69f13d8.el7.x86 3.4 MB/s | 4.7 MB 00:01 2024-04-25T20:44:00.325 INFO:teuthology.orchestra.run.smithi096.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.6 MB/s | 225 kB 00:00 2024-04-25T20:44:00.425 INFO:teuthology.orchestra.run.smithi096.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-25T20:44:00.509 INFO:teuthology.orchestra.run.smithi096.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-25T20:44:00.826 INFO:teuthology.orchestra.run.smithi096.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 2.3 MB/s | 747 kB 00:00 2024-04-25T20:44:00.959 INFO:teuthology.orchestra.run.smithi096.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.2 MB/s | 163 kB 00:00 2024-04-25T20:44:01.043 INFO:teuthology.orchestra.run.smithi096.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.5 MB/s | 126 kB 00:00 2024-04-25T20:44:01.310 INFO:teuthology.orchestra.run.smithi096.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 1.8 MB/s | 500 kB 00:00 2024-04-25T20:44:02.403 INFO:teuthology.orchestra.run.smithi097.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 5.1 MB/s | 11 MB 00:02 2024-04-25T20:44:02.495 INFO:teuthology.orchestra.run.smithi097.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 275 kB/s | 25 kB 00:00 2024-04-25T20:44:02.837 INFO:teuthology.orchestra.run.smithi097.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 3.2 MB/s | 21 MB 00:06 2024-04-25T20:44:02.842 INFO:teuthology.orchestra.run.smithi139.stdout:(8/152): ceph-radosgw-19.0.0-3156.gd69f13d8.el7 3.1 MB/s | 11 MB 00:03 2024-04-25T20:44:02.934 INFO:teuthology.orchestra.run.smithi139.stdout:(9/152): ceph-selinux-19.0.0-3156.gd69f13d8.el7 275 kB/s | 25 kB 00:00 2024-04-25T20:44:02.946 INFO:teuthology.orchestra.run.smithi097.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 225 kB 00:00 2024-04-25T20:44:03.046 INFO:teuthology.orchestra.run.smithi097.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 270 kB/s | 27 kB 00:00 2024-04-25T20:44:03.052 INFO:teuthology.orchestra.run.smithi096.stdout:(18/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 3.1 MB/s | 5.4 MB 00:01 2024-04-25T20:44:03.135 INFO:teuthology.orchestra.run.smithi096.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 543 kB/s | 45 kB 00:00 2024-04-25T20:44:03.138 INFO:teuthology.orchestra.run.smithi097.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 345 kB/s | 32 kB 00:00 2024-04-25T20:44:03.210 INFO:teuthology.orchestra.run.smithi096.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.8 MB/s | 135 kB 00:00 2024-04-25T20:44:03.302 INFO:teuthology.orchestra.run.smithi096.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.7 MB/s | 161 kB 00:00 2024-04-25T20:44:03.330 INFO:teuthology.orchestra.run.smithi097.stdout:(14/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 3.8 MB/s | 747 kB 00:00 2024-04-25T20:44:03.438 INFO:teuthology.orchestra.run.smithi097.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.5 MB/s | 163 kB 00:00 2024-04-25T20:44:03.461 INFO:teuthology.orchestra.run.smithi096.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.0 MB/s | 320 kB 00:00 2024-04-25T20:44:03.603 INFO:teuthology.orchestra.run.smithi096.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.1 MB/s | 298 kB 00:00 2024-04-25T20:44:03.605 INFO:teuthology.orchestra.run.smithi097.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 759 kB/s | 126 kB 00:00 2024-04-25T20:44:03.687 INFO:teuthology.orchestra.run.smithi096.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.2 MB/s | 99 kB 00:00 2024-04-25T20:44:03.762 INFO:teuthology.orchestra.run.smithi096.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T20:44:03.789 INFO:teuthology.orchestra.run.smithi097.stdout:(17/152): libradosstriper1-19.0.0-3156.gd69f13d 2.7 MB/s | 500 kB 00:00 2024-04-25T20:44:04.690 INFO:teuthology.orchestra.run.smithi097.stdout:(18/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 3.0 MB/s | 17 MB 00:05 2024-04-25T20:44:04.764 INFO:teuthology.orchestra.run.smithi097.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 604 kB/s | 45 kB 00:00 2024-04-25T20:44:04.849 INFO:teuthology.orchestra.run.smithi097.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.6 MB/s | 135 kB 00:00 2024-04-25T20:44:04.915 INFO:teuthology.orchestra.run.smithi097.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 2.4 MB/s | 161 kB 00:00 2024-04-25T20:44:04.934 INFO:teuthology.orchestra.run.smithi139.stdout:(10/152): ceph-common-19.0.0-3156.gd69f13d8.el7 2.4 MB/s | 21 MB 00:08 2024-04-25T20:44:04.962 INFO:teuthology.orchestra.run.smithi096.stdout:(26/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.6 MB/s | 3.1 MB 00:01 2024-04-25T20:44:04.999 INFO:teuthology.orchestra.run.smithi097.stdout:(22/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 4.4 MB/s | 5.4 MB 00:01 2024-04-25T20:44:05.029 INFO:teuthology.orchestra.run.smithi096.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 2.5 MB/s | 170 kB 00:00 2024-04-25T20:44:05.049 INFO:teuthology.orchestra.run.smithi097.stdout:(23/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.3 MB/s | 320 kB 00:00 2024-04-25T20:44:05.059 INFO:teuthology.orchestra.run.smithi139.stdout:(11/152): cephfs-mirror-19.0.0-3156.gd69f13d8.e 1.8 MB/s | 225 kB 00:00 2024-04-25T20:44:05.100 INFO:teuthology.orchestra.run.smithi097.stdout:(24/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 2.9 MB/s | 298 kB 00:00 2024-04-25T20:44:05.104 INFO:teuthology.orchestra.run.smithi096.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 457 kB/s | 34 kB 00:00 2024-04-25T20:44:05.125 INFO:teuthology.orchestra.run.smithi097.stdout:(25/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 1.3 MB/s | 99 kB 00:00 2024-04-25T20:44:05.142 INFO:teuthology.orchestra.run.smithi139.stdout:(12/152): cephfs-shell-19.0.0-3156.gd69f13d8.el 325 kB/s | 27 kB 00:00 2024-04-25T20:44:05.171 INFO:teuthology.orchestra.run.smithi096.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.1 MB/s | 142 kB 00:00 2024-04-25T20:44:05.175 INFO:teuthology.orchestra.run.smithi097.stdout:(26/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T20:44:05.226 INFO:teuthology.orchestra.run.smithi139.stdout:(13/152): libcephfs-devel-19.0.0-3156.gd69f13d8 381 kB/s | 32 kB 00:00 2024-04-25T20:44:05.275 INFO:teuthology.orchestra.run.smithi097.stdout:(27/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.7 MB/s | 170 kB 00:00 2024-04-25T20:44:05.367 INFO:teuthology.orchestra.run.smithi097.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3156.g 373 kB/s | 34 kB 00:00 2024-04-25T20:44:05.452 INFO:teuthology.orchestra.run.smithi139.stdout:(14/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 2.4 MB/s | 17 MB 00:06 2024-04-25T20:44:05.468 INFO:teuthology.orchestra.run.smithi097.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 1.4 MB/s | 142 kB 00:00 2024-04-25T20:44:05.543 INFO:teuthology.orchestra.run.smithi139.stdout:(15/152): libcephsqlite-19.0.0-3156.gd69f13d8.e 1.7 MB/s | 163 kB 00:00 2024-04-25T20:44:05.644 INFO:teuthology.orchestra.run.smithi139.stdout:(16/152): librados-devel-19.0.0-3156.gd69f13d8. 1.2 MB/s | 126 kB 00:00 2024-04-25T20:44:05.702 INFO:teuthology.orchestra.run.smithi139.stdout:(17/152): libcephfs2-19.0.0-3156.gd69f13d8.el7. 1.5 MB/s | 747 kB 00:00 2024-04-25T20:44:05.919 INFO:teuthology.orchestra.run.smithi139.stdout:(18/152): libradosstriper1-19.0.0-3156.gd69f13d 1.8 MB/s | 500 kB 00:00 2024-04-25T20:44:06.028 INFO:teuthology.orchestra.run.smithi139.stdout:(19/152): python3-ceph-argparse-19.0.0-3156.gd6 416 kB/s | 45 kB 00:00 2024-04-25T20:44:06.060 INFO:teuthology.orchestra.run.smithi097.stdout:(30/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 3.3 MB/s | 3.1 MB 00:00 2024-04-25T20:44:06.120 INFO:teuthology.orchestra.run.smithi139.stdout:(20/152): python3-ceph-common-19.0.0-3156.gd69f 1.4 MB/s | 135 kB 00:00 2024-04-25T20:44:06.228 INFO:teuthology.orchestra.run.smithi139.stdout:(21/152): python3-cephfs-19.0.0-3156.gd69f13d8. 1.5 MB/s | 161 kB 00:00 2024-04-25T20:44:06.235 INFO:teuthology.orchestra.run.smithi097.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.6 MB/s | 3.5 MB 00:00 2024-04-25T20:44:06.335 INFO:teuthology.orchestra.run.smithi097.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.4 MB/s | 249 kB 00:00 2024-04-25T20:44:06.379 INFO:teuthology.orchestra.run.smithi139.stdout:(22/152): python3-rados-19.0.0-3156.gd69f13d8.e 2.1 MB/s | 320 kB 00:00 2024-04-25T20:44:06.397 INFO:teuthology.orchestra.run.smithi096.stdout:(30/152): ceph-osd-19.0.0-3156.gd69f13d8.el7.x8 1.9 MB/s | 17 MB 00:08 2024-04-25T20:44:06.419 INFO:teuthology.orchestra.run.smithi097.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 592 kB/s | 49 kB 00:00 2024-04-25T20:44:06.463 INFO:teuthology.orchestra.run.smithi096.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 2.7 MB/s | 3.5 MB 00:01 2024-04-25T20:44:06.511 INFO:teuthology.orchestra.run.smithi097.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 179 kB/s | 16 kB 00:00 2024-04-25T20:44:06.537 INFO:teuthology.orchestra.run.smithi139.stdout:(23/152): python3-rbd-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 298 kB 00:00 2024-04-25T20:44:06.614 INFO:teuthology.orchestra.run.smithi096.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 1.6 MB/s | 249 kB 00:00 2024-04-25T20:44:06.628 INFO:teuthology.orchestra.run.smithi097.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 2.3 MB/s | 273 kB 00:00 2024-04-25T20:44:06.638 INFO:teuthology.orchestra.run.smithi139.stdout:(24/152): python3-rgw-19.0.0-3156.gd69f13d8.el7 995 kB/s | 99 kB 00:00 2024-04-25T20:44:06.689 INFO:teuthology.orchestra.run.smithi096.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 658 kB/s | 49 kB 00:00 2024-04-25T20:44:06.738 INFO:teuthology.orchestra.run.smithi139.stdout:(25/152): rbd-fuse-19.0.0-3156.gd69f13d8.el7.x8 853 kB/s | 85 kB 00:00 2024-04-25T20:44:06.772 INFO:teuthology.orchestra.run.smithi096.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 197 kB/s | 16 kB 00:00 2024-04-25T20:44:06.811 INFO:teuthology.orchestra.run.smithi097.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 4.1 MB/s | 762 kB 00:00 2024-04-25T20:44:06.912 INFO:teuthology.orchestra.run.smithi097.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 183 kB/s | 18 kB 00:00 2024-04-25T20:44:06.923 INFO:teuthology.orchestra.run.smithi096.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.8 MB/s | 273 kB 00:00 2024-04-25T20:44:07.012 INFO:teuthology.orchestra.run.smithi097.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 805 kB/s | 80 kB 00:00 2024-04-25T20:44:07.095 INFO:teuthology.orchestra.run.smithi097.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-04-25T20:44:07.138 INFO:teuthology.orchestra.run.smithi097.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T20:44:07.188 INFO:teuthology.orchestra.run.smithi097.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-25T20:44:07.256 INFO:teuthology.orchestra.run.smithi096.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 2.2 MB/s | 762 kB 00:00 2024-04-25T20:44:07.271 INFO:teuthology.orchestra.run.smithi097.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-25T20:44:07.305 INFO:teuthology.orchestra.run.smithi097.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-04-25T20:44:07.339 INFO:teuthology.orchestra.run.smithi097.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T20:44:07.356 INFO:teuthology.orchestra.run.smithi096.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 183 kB/s | 18 kB 00:00 2024-04-25T20:44:07.380 INFO:teuthology.orchestra.run.smithi097.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T20:44:07.422 INFO:teuthology.orchestra.run.smithi097.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-04-25T20:44:07.464 INFO:teuthology.orchestra.run.smithi097.stdout:(47/152): automake-1.16.2-8.el9.noarch.rpm 16 MB/s | 689 kB 00:00 2024-04-25T20:44:07.490 INFO:teuthology.orchestra.run.smithi096.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 603 kB/s | 80 kB 00:00 2024-04-25T20:44:07.523 INFO:teuthology.orchestra.run.smithi097.stdout:(48/152): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-04-25T20:44:07.557 INFO:teuthology.orchestra.run.smithi097.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 107 kB 00:00 2024-04-25T20:44:07.565 INFO:teuthology.orchestra.run.smithi096.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 10 MB/s | 803 kB 00:00 2024-04-25T20:44:07.582 INFO:teuthology.orchestra.run.smithi097.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 862 kB/s | 21 kB 00:00 2024-04-25T20:44:07.607 INFO:teuthology.orchestra.run.smithi096.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-25T20:44:07.624 INFO:teuthology.orchestra.run.smithi097.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 7.4 MB/s | 313 kB 00:00 2024-04-25T20:44:07.649 INFO:teuthology.orchestra.run.smithi096.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-25T20:44:07.658 INFO:teuthology.orchestra.run.smithi097.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 982 kB/s | 33 kB 00:00 2024-04-25T20:44:07.716 INFO:teuthology.orchestra.run.smithi096.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T20:44:07.766 INFO:teuthology.orchestra.run.smithi097.stdout:(53/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-04-25T20:44:07.771 INFO:teuthology.orchestra.run.smithi139.stdout:(26/152): librgw2-19.0.0-3156.gd69f13d8.el7.x86 2.6 MB/s | 5.4 MB 00:02 2024-04-25T20:44:07.775 INFO:teuthology.orchestra.run.smithi096.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-25T20:44:07.800 INFO:teuthology.orchestra.run.smithi097.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 549 kB/s | 18 kB 00:00 2024-04-25T20:44:07.808 INFO:teuthology.orchestra.run.smithi096.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 4.0 MB/s | 135 kB 00:00 2024-04-25T20:44:07.833 INFO:teuthology.orchestra.run.smithi097.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 337 kB/s | 11 kB 00:00 2024-04-25T20:44:07.851 INFO:teuthology.orchestra.run.smithi096.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T20:44:07.863 INFO:teuthology.orchestra.run.smithi139.stdout:(27/152): rbd-mirror-19.0.0-3156.gd69f13d8.el7. 2.7 MB/s | 3.1 MB 00:01 2024-04-25T20:44:07.875 INFO:teuthology.orchestra.run.smithi097.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-04-25T20:44:07.881 INFO:teuthology.orchestra.run.smithi139.stdout:(28/152): rbd-nbd-19.0.0-3156.gd69f13d8.el7.x86 1.5 MB/s | 170 kB 00:00 2024-04-25T20:44:07.917 INFO:teuthology.orchestra.run.smithi097.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T20:44:07.923 INFO:teuthology.orchestra.run.smithi139.stdout:(29/152): ceph-grafana-dashboards-19.0.0-3156.g 580 kB/s | 34 kB 00:00 2024-04-25T20:44:07.948 INFO:teuthology.orchestra.run.smithi139.stdout:(30/152): ceph-mgr-cephadm-19.0.0-3156.gd69f13d 2.1 MB/s | 142 kB 00:00 2024-04-25T20:44:08.002 INFO:teuthology.orchestra.run.smithi097.stdout:(58/152): ceph-mgr-diskprediction-local-19.0.0- 3.8 MB/s | 7.4 MB 00:01 2024-04-25T20:44:08.035 INFO:teuthology.orchestra.run.smithi097.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-25T20:44:08.060 INFO:teuthology.orchestra.run.smithi097.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T20:44:08.086 INFO:teuthology.orchestra.run.smithi097.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 517 kB/s | 25 kB 00:00 2024-04-25T20:44:08.111 INFO:teuthology.orchestra.run.smithi097.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-04-25T20:44:08.136 INFO:teuthology.orchestra.run.smithi097.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-04-25T20:44:08.161 INFO:teuthology.orchestra.run.smithi097.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-04-25T20:44:08.187 INFO:teuthology.orchestra.run.smithi097.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-04-25T20:44:08.293 INFO:teuthology.orchestra.run.smithi096.stdout:(46/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 5.0 MB/s | 49 MB 00:09 2024-04-25T20:44:08.312 INFO:teuthology.orchestra.run.smithi097.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 36 MB/s | 4.4 MB 00:00 2024-04-25T20:44:08.318 INFO:teuthology.orchestra.run.smithi096.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 460 kB/s | 215 kB 00:00 2024-04-25T20:44:08.351 INFO:teuthology.orchestra.run.smithi096.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 12 MB/s | 689 kB 00:00 2024-04-25T20:44:08.379 INFO:teuthology.orchestra.run.smithi097.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 22 MB/s | 4.7 MB 00:00 2024-04-25T20:44:08.393 INFO:teuthology.orchestra.run.smithi096.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-04-25T20:44:08.419 INFO:teuthology.orchestra.run.smithi096.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T20:44:08.421 INFO:teuthology.orchestra.run.smithi097.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T20:44:08.444 INFO:teuthology.orchestra.run.smithi096.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-04-25T20:44:08.463 INFO:teuthology.orchestra.run.smithi097.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T20:44:08.469 INFO:teuthology.orchestra.run.smithi096.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T20:44:08.486 INFO:teuthology.orchestra.run.smithi096.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 797 kB/s | 33 kB 00:00 2024-04-25T20:44:08.488 INFO:teuthology.orchestra.run.smithi097.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T20:44:08.520 INFO:teuthology.orchestra.run.smithi096.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 553 kB/s | 18 kB 00:00 2024-04-25T20:44:08.580 INFO:teuthology.orchestra.run.smithi097.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-25T20:44:08.620 INFO:teuthology.orchestra.run.smithi096.stdout:(55/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-04-25T20:44:08.637 INFO:teuthology.orchestra.run.smithi096.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 96 kB/s | 11 kB 00:00 2024-04-25T20:44:08.662 INFO:teuthology.orchestra.run.smithi096.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T20:44:08.663 INFO:teuthology.orchestra.run.smithi097.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T20:44:08.688 INFO:teuthology.orchestra.run.smithi096.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T20:44:08.689 INFO:teuthology.orchestra.run.smithi097.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 825 kB/s | 87 kB 00:00 2024-04-25T20:44:08.707 INFO:teuthology.orchestra.run.smithi139.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3156.gd69f1 4.5 MB/s | 3.5 MB 00:00 2024-04-25T20:44:08.713 INFO:teuthology.orchestra.run.smithi096.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T20:44:08.739 INFO:teuthology.orchestra.run.smithi097.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.9 MB/s | 244 kB 00:00 2024-04-25T20:44:08.746 INFO:teuthology.orchestra.run.smithi096.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 9.9 MB/s | 582 kB 00:00 2024-04-25T20:44:08.780 INFO:teuthology.orchestra.run.smithi096.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 386 kB/s | 25 kB 00:00 2024-04-25T20:44:08.781 INFO:teuthology.orchestra.run.smithi097.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 6.1 MB/s | 248 kB 00:00 2024-04-25T20:44:08.806 INFO:teuthology.orchestra.run.smithi096.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 811 kB/s | 47 kB 00:00 2024-04-25T20:44:08.807 INFO:teuthology.orchestra.run.smithi097.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 1.1 MB/s | 26 kB 00:00 2024-04-25T20:44:08.824 INFO:teuthology.orchestra.run.smithi139.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3156.gd6 2.1 MB/s | 249 kB 00:00 2024-04-25T20:44:08.839 INFO:teuthology.orchestra.run.smithi096.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T20:44:08.864 INFO:teuthology.orchestra.run.smithi096.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T20:44:08.881 INFO:teuthology.orchestra.run.smithi097.stdout:(77/152): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2024-04-25T20:44:08.889 INFO:teuthology.orchestra.run.smithi096.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 708 kB/s | 35 kB 00:00 2024-04-25T20:44:08.890 INFO:teuthology.orchestra.run.smithi139.stdout:(33/152): ceph-mgr-rook-19.0.0-3156.gd69f13d8.e 742 kB/s | 49 kB 00:00 2024-04-25T20:44:08.898 INFO:teuthology.orchestra.run.smithi097.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-04-25T20:44:08.924 INFO:teuthology.orchestra.run.smithi097.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-04-25T20:44:08.949 INFO:teuthology.orchestra.run.smithi097.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 697 kB/s | 35 kB 00:00 2024-04-25T20:44:08.966 INFO:teuthology.orchestra.run.smithi139.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3156.gd 219 kB/s | 16 kB 00:00 2024-04-25T20:44:09.024 INFO:teuthology.orchestra.run.smithi097.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T20:44:09.040 INFO:teuthology.orchestra.run.smithi096.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2024-04-25T20:44:09.107 INFO:teuthology.orchestra.run.smithi096.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-25T20:44:09.107 INFO:teuthology.orchestra.run.smithi139.stdout:(35/152): ceph-volume-19.0.0-3156.gd69f13d8.el7 1.9 MB/s | 273 kB 00:00 2024-04-25T20:44:09.108 INFO:teuthology.orchestra.run.smithi097.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 28 MB/s | 5.1 MB 00:00 2024-04-25T20:44:09.133 INFO:teuthology.orchestra.run.smithi097.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-04-25T20:44:09.149 INFO:teuthology.orchestra.run.smithi096.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T20:44:09.150 INFO:teuthology.orchestra.run.smithi097.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.8 MB/s | 77 kB 00:00 2024-04-25T20:44:09.176 INFO:teuthology.orchestra.run.smithi097.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-04-25T20:44:09.182 INFO:teuthology.orchestra.run.smithi096.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T20:44:09.201 INFO:teuthology.orchestra.run.smithi097.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T20:44:09.208 INFO:teuthology.orchestra.run.smithi096.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T20:44:09.234 INFO:teuthology.orchestra.run.smithi097.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T20:44:09.260 INFO:teuthology.orchestra.run.smithi097.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-04-25T20:44:09.283 INFO:teuthology.orchestra.run.smithi096.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2024-04-25T20:44:09.285 INFO:teuthology.orchestra.run.smithi097.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T20:44:09.310 INFO:teuthology.orchestra.run.smithi097.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T20:44:09.316 INFO:teuthology.orchestra.run.smithi139.stdout:(36/152): cephadm-19.0.0-3156.gd69f13d8.el7.noa 3.6 MB/s | 762 kB 00:00 2024-04-25T20:44:09.352 INFO:teuthology.orchestra.run.smithi097.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-25T20:44:09.375 INFO:teuthology.orchestra.run.smithi096.stdout:(72/152): ceph-mgr-diskprediction-local-19.0.0- 2.5 MB/s | 7.4 MB 00:02 2024-04-25T20:44:09.383 INFO:teuthology.orchestra.run.smithi139.stdout:(37/152): cephfs-top-19.0.0-3156.gd69f13d8.el7. 274 kB/s | 18 kB 00:00 2024-04-25T20:44:09.394 INFO:teuthology.orchestra.run.smithi097.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-25T20:44:09.450 INFO:teuthology.orchestra.run.smithi096.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-04-25T20:44:09.453 INFO:teuthology.orchestra.run.smithi097.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.9 MB/s | 221 kB 00:00 2024-04-25T20:44:09.475 INFO:teuthology.orchestra.run.smithi096.stdout:(74/152): python3-attrs-20.3.0-7.el9.noarch.rpm 461 kB/s | 87 kB 00:00 2024-04-25T20:44:09.511 INFO:teuthology.orchestra.run.smithi097.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 641 kB/s | 36 kB 00:00 2024-04-25T20:44:09.578 INFO:teuthology.orchestra.run.smithi097.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 730 kB/s | 47 kB 00:00 2024-04-25T20:44:09.592 INFO:teuthology.orchestra.run.smithi096.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 28 MB/s | 6.0 MB 00:00 2024-04-25T20:44:09.600 INFO:teuthology.orchestra.run.smithi139.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 371 kB/s | 80 kB 00:00 2024-04-25T20:44:09.617 INFO:teuthology.orchestra.run.smithi096.stdout:(76/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.7 MB/s | 244 kB 00:00 2024-04-25T20:44:09.643 INFO:teuthology.orchestra.run.smithi096.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-04-25T20:44:09.668 INFO:teuthology.orchestra.run.smithi096.stdout:(78/152): python3-jsonpatch-1.21-16.el9.noarch. 344 kB/s | 26 kB 00:00 2024-04-25T20:44:09.692 INFO:teuthology.orchestra.run.smithi139.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 8.6 MB/s | 803 kB 00:00 2024-04-25T20:44:09.693 INFO:teuthology.orchestra.run.smithi096.stdout:(79/152): python3-jsonpointer-2.0-4.el9.noarch. 384 kB/s | 19 kB 00:00 2024-04-25T20:44:09.718 INFO:teuthology.orchestra.run.smithi096.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-25T20:44:09.743 INFO:teuthology.orchestra.run.smithi096.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-04-25T20:44:09.745 INFO:teuthology.orchestra.run.smithi097.stdout:(96/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T20:44:09.769 INFO:teuthology.orchestra.run.smithi096.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-04-25T20:44:09.775 INFO:teuthology.orchestra.run.smithi139.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-25T20:44:09.779 INFO:teuthology.orchestra.run.smithi097.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 305 kB 00:00 2024-04-25T20:44:09.794 INFO:teuthology.orchestra.run.smithi096.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-04-25T20:44:09.800 INFO:teuthology.orchestra.run.smithi139.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.8 MB/s | 195 kB 00:00 2024-04-25T20:44:09.804 INFO:teuthology.orchestra.run.smithi097.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 998 kB/s | 58 kB 00:00 2024-04-25T20:44:09.819 INFO:teuthology.orchestra.run.smithi096.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.6 MB/s | 77 kB 00:00 2024-04-25T20:44:09.826 INFO:teuthology.orchestra.run.smithi139.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T20:44:09.829 INFO:teuthology.orchestra.run.smithi097.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-25T20:44:09.855 INFO:teuthology.orchestra.run.smithi097.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-25T20:44:09.868 INFO:teuthology.orchestra.run.smithi139.stdout:(43/152): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-25T20:44:09.880 INFO:teuthology.orchestra.run.smithi097.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-04-25T20:44:09.893 INFO:teuthology.orchestra.run.smithi139.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-25T20:44:09.895 INFO:teuthology.orchestra.run.smithi096.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-04-25T20:44:09.905 INFO:teuthology.orchestra.run.smithi097.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 446 kB/s | 22 kB 00:00 2024-04-25T20:44:09.918 INFO:teuthology.orchestra.run.smithi139.stdout:(45/152): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-25T20:44:09.919 INFO:teuthology.orchestra.run.smithi096.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 342 kB/s | 42 kB 00:00 2024-04-25T20:44:09.939 INFO:teuthology.orchestra.run.smithi097.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 14 MB/s | 863 kB 00:00 2024-04-25T20:44:09.944 INFO:teuthology.orchestra.run.smithi139.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-25T20:44:09.945 INFO:teuthology.orchestra.run.smithi096.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 2.1 MB/s | 269 kB 00:00 2024-04-25T20:44:09.969 INFO:teuthology.orchestra.run.smithi139.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-25T20:44:09.970 INFO:teuthology.orchestra.run.smithi096.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T20:44:09.994 INFO:teuthology.orchestra.run.smithi139.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-04-25T20:44:10.004 INFO:teuthology.orchestra.run.smithi096.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T20:44:10.029 INFO:teuthology.orchestra.run.smithi096.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 610 kB/s | 51 kB 00:00 2024-04-25T20:44:10.036 INFO:teuthology.orchestra.run.smithi139.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 943 kB 00:00 2024-04-25T20:44:10.054 INFO:teuthology.orchestra.run.smithi096.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T20:44:10.061 INFO:teuthology.orchestra.run.smithi139.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-04-25T20:44:10.078 INFO:teuthology.orchestra.run.smithi139.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 515 kB/s | 21 kB 00:00 2024-04-25T20:44:10.079 INFO:teuthology.orchestra.run.smithi096.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 830 kB/s | 42 kB 00:00 2024-04-25T20:44:10.104 INFO:teuthology.orchestra.run.smithi139.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-04-25T20:44:10.105 INFO:teuthology.orchestra.run.smithi096.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 905 kB/s | 44 kB 00:00 2024-04-25T20:44:10.121 INFO:teuthology.orchestra.run.smithi139.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 792 kB/s | 33 kB 00:00 2024-04-25T20:44:10.139 INFO:teuthology.orchestra.run.smithi096.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 3.8 MB/s | 221 kB 00:00 2024-04-25T20:44:10.154 INFO:teuthology.orchestra.run.smithi139.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 555 kB/s | 18 kB 00:00 2024-04-25T20:44:10.164 INFO:teuthology.orchestra.run.smithi096.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 639 kB/s | 36 kB 00:00 2024-04-25T20:44:10.180 INFO:teuthology.orchestra.run.smithi139.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 482 kB/s | 11 kB 00:00 2024-04-25T20:44:10.189 INFO:teuthology.orchestra.run.smithi096.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 992 kB/s | 47 kB 00:00 2024-04-25T20:44:10.214 INFO:teuthology.orchestra.run.smithi096.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 6.1 MB/s | 305 kB 00:00 2024-04-25T20:44:10.230 INFO:teuthology.orchestra.run.smithi139.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T20:44:10.231 INFO:teuthology.orchestra.run.smithi096.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-04-25T20:44:10.255 INFO:teuthology.orchestra.run.smithi139.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-25T20:44:10.256 INFO:teuthology.orchestra.run.smithi096.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.3 MB/s | 94 kB 00:00 2024-04-25T20:44:10.273 INFO:teuthology.orchestra.run.smithi096.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.6 MB/s | 64 kB 00:00 2024-04-25T20:44:10.280 INFO:teuthology.orchestra.run.smithi139.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-04-25T20:44:10.299 INFO:teuthology.orchestra.run.smithi096.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-25T20:44:10.306 INFO:teuthology.orchestra.run.smithi139.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-04-25T20:44:10.316 INFO:teuthology.orchestra.run.smithi096.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 567 kB/s | 22 kB 00:00 2024-04-25T20:44:10.339 INFO:teuthology.orchestra.run.smithi139.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 10 MB/s | 582 kB 00:00 2024-04-25T20:44:10.365 INFO:teuthology.orchestra.run.smithi139.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 444 kB/s | 25 kB 00:00 2024-04-25T20:44:10.390 INFO:teuthology.orchestra.run.smithi139.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-25T20:44:10.424 INFO:teuthology.orchestra.run.smithi139.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T20:44:10.449 INFO:teuthology.orchestra.run.smithi139.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-04-25T20:44:10.474 INFO:teuthology.orchestra.run.smithi139.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 709 kB/s | 35 kB 00:00 2024-04-25T20:44:10.475 INFO:teuthology.orchestra.run.smithi096.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-04-25T20:44:10.516 INFO:teuthology.orchestra.run.smithi096.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-04-25T20:44:10.523 INFO:teuthology.orchestra.run.smithi097.stdout:(104/152): ceph-test-19.0.0-3156.gd69f13d8.el7. 6.1 MB/s | 49 MB 00:08 2024-04-25T20:44:10.617 INFO:teuthology.orchestra.run.smithi096.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-04-25T20:44:10.641 INFO:teuthology.orchestra.run.smithi139.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-04-25T20:44:10.667 INFO:teuthology.orchestra.run.smithi096.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.6 MB/s | 309 kB 00:00 2024-04-25T20:44:10.681 INFO:teuthology.orchestra.run.smithi097.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 708 kB/s | 548 kB 00:00 2024-04-25T20:44:10.700 INFO:teuthology.orchestra.run.smithi096.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.4 MB/s | 548 kB 00:00 2024-04-25T20:44:10.708 INFO:teuthology.orchestra.run.smithi139.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-25T20:44:10.714 INFO:teuthology.orchestra.run.smithi097.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 102 kB/s | 19 kB 00:00 2024-04-25T20:44:10.726 INFO:teuthology.orchestra.run.smithi096.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 444 kB/s | 26 kB 00:00 2024-04-25T20:44:10.750 INFO:teuthology.orchestra.run.smithi139.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-04-25T20:44:10.751 INFO:teuthology.orchestra.run.smithi096.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 969 kB/s | 49 kB 00:00 2024-04-25T20:44:10.773 INFO:teuthology.orchestra.run.smithi097.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 370 kB/s | 309 kB 00:00 2024-04-25T20:44:10.776 INFO:teuthology.orchestra.run.smithi096.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-25T20:44:10.783 INFO:teuthology.orchestra.run.smithi139.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-04-25T20:44:10.799 INFO:teuthology.orchestra.run.smithi097.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 310 kB/s | 26 kB 00:00 2024-04-25T20:44:10.801 INFO:teuthology.orchestra.run.smithi096.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-04-25T20:44:10.808 INFO:teuthology.orchestra.run.smithi139.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.3 MB/s | 133 kB 00:00 2024-04-25T20:44:10.868 INFO:teuthology.orchestra.run.smithi096.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.9 MB/s | 838 kB 00:00 2024-04-25T20:44:10.892 INFO:teuthology.orchestra.run.smithi139.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-04-25T20:44:10.902 INFO:teuthology.orchestra.run.smithi096.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-04-25T20:44:10.924 INFO:teuthology.orchestra.run.smithi097.stdout:(109/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T20:44:10.944 INFO:teuthology.orchestra.run.smithi096.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 577 kB/s | 43 kB 00:00 2024-04-25T20:44:10.949 INFO:teuthology.orchestra.run.smithi097.stdout:(110/152): liboath-2.6.7-2.el9.x86_64.rpm 277 kB/s | 49 kB 00:00 2024-04-25T20:44:10.969 INFO:teuthology.orchestra.run.smithi096.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 482 kB/s | 32 kB 00:00 2024-04-25T20:44:10.982 INFO:teuthology.orchestra.run.smithi097.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 366 kB/s | 67 kB 00:00 2024-04-25T20:44:10.984 INFO:teuthology.orchestra.run.smithi139.stdout:(72/152): postgresql-server-13.14-1.el9.x86_64. 29 MB/s | 5.8 MB 00:00 2024-04-25T20:44:10.994 INFO:teuthology.orchestra.run.smithi096.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T20:44:11.001 INFO:teuthology.orchestra.run.smithi139.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 825 kB/s | 87 kB 00:00 2024-04-25T20:44:11.008 INFO:teuthology.orchestra.run.smithi097.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.8 MB/s | 151 kB 00:00 2024-04-25T20:44:11.020 INFO:teuthology.orchestra.run.smithi096.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T20:44:11.043 INFO:teuthology.orchestra.run.smithi139.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 5.8 MB/s | 244 kB 00:00 2024-04-25T20:44:11.045 INFO:teuthology.orchestra.run.smithi096.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-25T20:44:11.058 INFO:teuthology.orchestra.run.smithi097.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 870 kB/s | 43 kB 00:00 2024-04-25T20:44:11.070 INFO:teuthology.orchestra.run.smithi096.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-25T20:44:11.085 INFO:teuthology.orchestra.run.smithi139.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 6.2 MB/s | 248 kB 00:00 2024-04-25T20:44:11.096 INFO:teuthology.orchestra.run.smithi096.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-04-25T20:44:11.108 INFO:teuthology.orchestra.run.smithi097.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 645 kB/s | 32 kB 00:00 2024-04-25T20:44:11.121 INFO:teuthology.orchestra.run.smithi096.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T20:44:11.134 INFO:teuthology.orchestra.run.smithi097.stdout:(115/152): python3-asyncssh-2.13.2-5.el9.noarch 3.6 MB/s | 548 kB 00:00 2024-04-25T20:44:11.159 INFO:teuthology.orchestra.run.smithi097.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T20:44:11.160 INFO:teuthology.orchestra.run.smithi139.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-04-25T20:44:11.185 INFO:teuthology.orchestra.run.smithi097.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T20:44:11.186 INFO:teuthology.orchestra.run.smithi139.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 262 kB/s | 26 kB 00:00 2024-04-25T20:44:11.205 INFO:teuthology.orchestra.run.smithi096.stdout:(122/152): libarrow-9.0.0-10.el9.x86_64.rpm 7.5 MB/s | 4.4 MB 00:00 2024-04-25T20:44:11.211 INFO:teuthology.orchestra.run.smithi139.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-04-25T20:44:11.226 INFO:teuthology.orchestra.run.smithi097.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-04-25T20:44:11.230 INFO:teuthology.orchestra.run.smithi096.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-25T20:44:11.236 INFO:teuthology.orchestra.run.smithi139.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-04-25T20:44:11.251 INFO:teuthology.orchestra.run.smithi097.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-04-25T20:44:11.255 INFO:teuthology.orchestra.run.smithi096.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-25T20:44:11.261 INFO:teuthology.orchestra.run.smithi139.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 700 kB/s | 35 kB 00:00 2024-04-25T20:44:11.277 INFO:teuthology.orchestra.run.smithi097.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 703 kB/s | 35 kB 00:00 2024-04-25T20:44:11.280 INFO:teuthology.orchestra.run.smithi096.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-04-25T20:44:11.302 INFO:teuthology.orchestra.run.smithi097.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T20:44:11.305 INFO:teuthology.orchestra.run.smithi139.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.8 MB/s | 250 kB 00:00 2024-04-25T20:44:11.306 INFO:teuthology.orchestra.run.smithi096.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-25T20:44:11.331 INFO:teuthology.orchestra.run.smithi096.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-04-25T20:44:11.352 INFO:teuthology.orchestra.run.smithi097.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-04-25T20:44:11.356 INFO:teuthology.orchestra.run.smithi096.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-04-25T20:44:11.407 INFO:teuthology.orchestra.run.smithi096.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-25T20:44:11.411 INFO:teuthology.orchestra.run.smithi097.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-25T20:44:11.428 INFO:teuthology.orchestra.run.smithi097.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 142 kB/s | 11 kB 00:00 2024-04-25T20:44:11.432 INFO:teuthology.orchestra.run.smithi096.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 616 kB/s | 46 kB 00:00 2024-04-25T20:44:11.457 INFO:teuthology.orchestra.run.smithi096.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-04-25T20:44:11.461 INFO:teuthology.orchestra.run.smithi097.stdout:(125/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.6 MB/s | 838 kB 00:00 2024-04-25T20:44:11.467 INFO:teuthology.orchestra.run.smithi139.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 22 MB/s | 5.1 MB 00:00 2024-04-25T20:44:11.478 INFO:teuthology.orchestra.run.smithi097.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 263 kB/s | 18 kB 00:00 2024-04-25T20:44:11.482 INFO:teuthology.orchestra.run.smithi096.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-04-25T20:44:11.495 INFO:teuthology.orchestra.run.smithi097.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 345 kB/s | 23 kB 00:00 2024-04-25T20:44:11.506 INFO:teuthology.orchestra.run.smithi139.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.1 MB/s | 222 kB 00:00 2024-04-25T20:44:11.512 INFO:teuthology.orchestra.run.smithi097.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-25T20:44:11.516 INFO:teuthology.orchestra.run.smithi096.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 5.5 MB/s | 1.0 MB 00:00 2024-04-25T20:44:11.531 INFO:teuthology.orchestra.run.smithi139.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 77 kB 00:00 2024-04-25T20:44:11.537 INFO:teuthology.orchestra.run.smithi097.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 473 kB/s | 20 kB 00:00 2024-04-25T20:44:11.541 INFO:teuthology.orchestra.run.smithi096.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-25T20:44:11.556 INFO:teuthology.orchestra.run.smithi139.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 845 kB/s | 42 kB 00:00 2024-04-25T20:44:11.566 INFO:teuthology.orchestra.run.smithi096.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-25T20:44:11.571 INFO:teuthology.orchestra.run.smithi097.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 791 kB/s | 46 kB 00:00 2024-04-25T20:44:11.590 INFO:teuthology.orchestra.run.smithi139.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-04-25T20:44:11.591 INFO:teuthology.orchestra.run.smithi096.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-04-25T20:44:11.596 INFO:teuthology.orchestra.run.smithi097.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-04-25T20:44:11.615 INFO:teuthology.orchestra.run.smithi139.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-04-25T20:44:11.617 INFO:teuthology.orchestra.run.smithi096.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-04-25T20:44:11.630 INFO:teuthology.orchestra.run.smithi097.stdout:(132/152): python3-kubernetes-26.1.0-2.el9.noar 7.6 MB/s | 1.0 MB 00:00 2024-04-25T20:44:11.634 INFO:teuthology.orchestra.run.smithi096.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 765 kB/s | 90 kB 00:00 2024-04-25T20:44:11.641 INFO:teuthology.orchestra.run.smithi139.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.5 MB/s | 279 kB 00:00 2024-04-25T20:44:11.655 INFO:teuthology.orchestra.run.smithi097.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 689 kB/s | 58 kB 00:00 2024-04-25T20:44:11.659 INFO:teuthology.orchestra.run.smithi096.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-25T20:44:11.666 INFO:teuthology.orchestra.run.smithi139.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T20:44:11.680 INFO:teuthology.orchestra.run.smithi097.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-04-25T20:44:11.684 INFO:teuthology.orchestra.run.smithi096.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-04-25T20:44:11.691 INFO:teuthology.orchestra.run.smithi139.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-25T20:44:11.710 INFO:teuthology.orchestra.run.smithi096.stdout:(141/152): python3-rsa-4.9-2.el9.noarch.rpm 640 kB/s | 59 kB 00:00 2024-04-25T20:44:11.714 INFO:teuthology.orchestra.run.smithi097.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-04-25T20:44:11.733 INFO:teuthology.orchestra.run.smithi139.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-04-25T20:44:11.735 INFO:teuthology.orchestra.run.smithi096.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-25T20:44:11.760 INFO:teuthology.orchestra.run.smithi096.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-25T20:44:11.775 INFO:teuthology.orchestra.run.smithi139.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.1 MB/s | 44 kB 00:00 2024-04-25T20:44:11.785 INFO:teuthology.orchestra.run.smithi096.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T20:44:11.802 INFO:teuthology.orchestra.run.smithi096.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-04-25T20:44:11.828 INFO:teuthology.orchestra.run.smithi096.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-04-25T20:44:11.840 INFO:teuthology.orchestra.run.smithi097.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 178 kB/s | 28 kB 00:00 2024-04-25T20:44:11.853 INFO:teuthology.orchestra.run.smithi096.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T20:44:11.864 INFO:teuthology.orchestra.run.smithi097.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 205 kB/s | 31 kB 00:00 2024-04-25T20:44:11.890 INFO:teuthology.orchestra.run.smithi097.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 384 kB/s | 90 kB 00:00 2024-04-25T20:44:11.915 INFO:teuthology.orchestra.run.smithi097.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-04-25T20:44:11.937 INFO:teuthology.orchestra.run.smithi096.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T20:44:11.940 INFO:teuthology.orchestra.run.smithi097.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-25T20:44:11.966 INFO:teuthology.orchestra.run.smithi097.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-04-25T20:44:11.991 INFO:teuthology.orchestra.run.smithi097.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 677 kB/s | 51 kB 00:00 2024-04-25T20:44:12.003 INFO:teuthology.orchestra.run.smithi096.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-04-25T20:44:12.016 INFO:teuthology.orchestra.run.smithi097.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-04-25T20:44:12.041 INFO:teuthology.orchestra.run.smithi097.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-25T20:44:12.067 INFO:teuthology.orchestra.run.smithi097.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-25T20:44:12.092 INFO:teuthology.orchestra.run.smithi097.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T20:44:12.095 INFO:teuthology.orchestra.run.smithi096.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T20:44:12.201 INFO:teuthology.orchestra.run.smithi097.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.0 MB/s | 427 kB 00:00 2024-04-25T20:44:12.209 INFO:teuthology.orchestra.run.smithi139.stdout:(93/152): ceph-test-19.0.0-3156.gd69f13d8.el7.x 5.3 MB/s | 49 MB 00:09 2024-04-25T20:44:12.234 INFO:teuthology.orchestra.run.smithi139.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 484 kB/s | 221 kB 00:00 2024-04-25T20:44:12.243 INFO:teuthology.orchestra.run.smithi097.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-04-25T20:44:12.327 INFO:teuthology.orchestra.run.smithi097.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 16 MB/s | 1.3 MB 00:00 2024-04-25T20:44:12.418 INFO:teuthology.orchestra.run.smithi097.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-04-25T20:44:12.429 INFO:teuthology.orchestra.run.smithi096.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 5.7 MB/s | 3.4 MB 00:00 2024-04-25T20:44:12.451 INFO:teuthology.orchestra.run.smithi139.stdout:(95/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 21 MB/s | 16 MB 00:00 2024-04-25T20:44:12.477 INFO:teuthology.orchestra.run.smithi139.stdout:(96/152): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 36 kB 00:00 2024-04-25T20:44:12.501 INFO:teuthology.orchestra.run.smithi139.stdout:(97/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 179 kB/s | 47 kB 00:00 2024-04-25T20:44:12.527 INFO:teuthology.orchestra.run.smithi139.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T20:44:12.552 INFO:teuthology.orchestra.run.smithi139.stdout:(99/152): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-25T20:44:12.577 INFO:teuthology.orchestra.run.smithi139.stdout:(100/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.2 MB/s | 94 kB 00:00 2024-04-25T20:44:12.602 INFO:teuthology.orchestra.run.smithi139.stdout:(101/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-04-25T20:44:12.628 INFO:teuthology.orchestra.run.smithi139.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 447 kB/s | 22 kB 00:00 2024-04-25T20:44:12.653 INFO:teuthology.orchestra.run.smithi139.stdout:(103/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 596 kB/s | 45 kB 00:00 2024-04-25T20:44:12.660 INFO:teuthology.orchestra.run.smithi097.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 6.0 MB/s | 3.4 MB 00:00 2024-04-25T20:44:12.679 INFO:teuthology.orchestra.run.smithi096.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 3.8 MB/s | 3.1 MB 00:00 2024-04-25T20:44:12.683 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:44:12.686 INFO:teuthology.orchestra.run.smithi096.stdout:Total 13 MB/s | 222 MB 00:17 2024-04-25T20:44:12.686 INFO:teuthology.orchestra.run.smithi139.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 10 MB/s | 863 kB 00:00 2024-04-25T20:44:12.710 INFO:teuthology.orchestra.run.smithi097.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 6.2 MB/s | 3.1 MB 00:00 2024-04-25T20:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:44:12.717 INFO:teuthology.orchestra.run.smithi097.stdout:Total 13 MB/s | 222 MB 00:16 2024-04-25T20:44:12.837 INFO:teuthology.orchestra.run.smithi139.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 130 kB/s | 19 kB 00:00 2024-04-25T20:44:12.928 INFO:teuthology.orchestra.run.smithi139.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-04-25T20:44:12.953 INFO:teuthology.orchestra.run.smithi139.stdout:(107/152): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-04-25T20:44:13.004 INFO:teuthology.orchestra.run.smithi139.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-04-25T20:44:13.029 INFO:teuthology.orchestra.run.smithi139.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 650 kB/s | 49 kB 00:00 2024-04-25T20:44:13.096 INFO:teuthology.orchestra.run.smithi139.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T20:44:13.121 INFO:teuthology.orchestra.run.smithi139.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 582 kB/s | 67 kB 00:00 2024-04-25T20:44:13.146 INFO:teuthology.orchestra.run.smithi139.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-04-25T20:44:13.180 INFO:teuthology.orchestra.run.smithi139.stdout:(113/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 14 MB/s | 838 kB 00:00 2024-04-25T20:44:13.205 INFO:teuthology.orchestra.run.smithi139.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 6.4 MB/s | 548 kB 00:00 2024-04-25T20:44:13.230 INFO:teuthology.orchestra.run.smithi139.stdout:(115/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 526 kB/s | 43 kB 00:00 2024-04-25T20:44:13.256 INFO:teuthology.orchestra.run.smithi139.stdout:(116/152): python3-cachetools-4.2.4-1.el9.noarc 433 kB/s | 32 kB 00:00 2024-04-25T20:44:13.281 INFO:teuthology.orchestra.run.smithi139.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-04-25T20:44:13.306 INFO:teuthology.orchestra.run.smithi139.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-25T20:44:13.332 INFO:teuthology.orchestra.run.smithi139.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 4.7 MB/s | 358 kB 00:00 2024-04-25T20:44:13.373 INFO:teuthology.orchestra.run.smithi139.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.5 MB/s | 303 kB 00:00 2024-04-25T20:44:13.399 INFO:teuthology.orchestra.run.smithi139.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 387 kB/s | 35 kB 00:00 2024-04-25T20:44:13.425 INFO:teuthology.orchestra.run.smithi139.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-04-25T20:44:13.466 INFO:teuthology.orchestra.run.smithi139.stdout:(123/152): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-04-25T20:44:13.491 INFO:teuthology.orchestra.run.smithi139.stdout:(124/152): python3-grpcio-tools-1.46.7-10.el9.x 2.2 MB/s | 144 kB 00:00 2024-04-25T20:44:13.508 INFO:teuthology.orchestra.run.smithi139.stdout:(125/152): python3-jaraco-8.2.1-3.el9.noarch.rp 131 kB/s | 11 kB 00:00 2024-04-25T20:44:13.533 INFO:teuthology.orchestra.run.smithi139.stdout:(126/152): python3-jaraco-classes-3.2.1-5.el9.n 265 kB/s | 18 kB 00:00 2024-04-25T20:44:13.558 INFO:teuthology.orchestra.run.smithi139.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 463 kB/s | 23 kB 00:00 2024-04-25T20:44:13.584 INFO:teuthology.orchestra.run.smithi139.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 259 kB/s | 19 kB 00:00 2024-04-25T20:44:13.609 INFO:teuthology.orchestra.run.smithi139.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 262 kB/s | 20 kB 00:00 2024-04-25T20:44:13.651 INFO:teuthology.orchestra.run.smithi139.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 15 MB/s | 1.0 MB 00:00 2024-04-25T20:44:13.668 INFO:teuthology.orchestra.run.smithi139.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 556 kB/s | 46 kB 00:00 2024-04-25T20:44:13.693 INFO:teuthology.orchestra.run.smithi139.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 956 kB/s | 79 kB 00:00 2024-04-25T20:44:13.710 INFO:teuthology.orchestra.run.smithi139.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 988 kB/s | 58 kB 00:00 2024-04-25T20:44:13.735 INFO:teuthology.orchestra.run.smithi139.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 6.4 MB/s | 272 kB 00:00 2024-04-25T20:44:13.752 INFO:teuthology.orchestra.run.smithi139.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 282 kB/s | 16 kB 00:00 2024-04-25T20:44:13.776 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2024-04-25T20:44:13.777 INFO:teuthology.orchestra.run.smithi139.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.3 MB/s | 90 kB 00:00 2024-04-25T20:44:13.791 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-04-25T20:44:13.802 INFO:teuthology.orchestra.run.smithi139.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 424 kB/s | 28 kB 00:00 2024-04-25T20:44:13.827 INFO:teuthology.orchestra.run.smithi139.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-04-25T20:44:13.853 INFO:teuthology.orchestra.run.smithi139.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-04-25T20:44:13.878 INFO:teuthology.orchestra.run.smithi139.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-25T20:44:13.903 INFO:teuthology.orchestra.run.smithi139.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-25T20:44:13.918 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2024-04-25T20:44:13.918 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2024-04-25T20:44:13.928 INFO:teuthology.orchestra.run.smithi139.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 681 kB/s | 51 kB 00:00 2024-04-25T20:44:13.932 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-04-25T20:44:13.932 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-04-25T20:44:13.954 INFO:teuthology.orchestra.run.smithi139.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-04-25T20:44:13.979 INFO:teuthology.orchestra.run.smithi139.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-25T20:44:14.004 INFO:teuthology.orchestra.run.smithi139.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-04-25T20:44:14.029 INFO:teuthology.orchestra.run.smithi139.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 406 kB/s | 20 kB 00:00 2024-04-25T20:44:14.055 INFO:teuthology.orchestra.run.smithi139.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-04-25T20:44:14.138 INFO:teuthology.orchestra.run.smithi139.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-04-25T20:44:14.213 INFO:teuthology.orchestra.run.smithi139.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 18 MB/s | 1.3 MB 00:00 2024-04-25T20:44:14.331 INFO:teuthology.orchestra.run.smithi139.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 20 MB/s | 2.4 MB 00:00 2024-04-25T20:44:14.489 INFO:teuthology.orchestra.run.smithi139.stdout:(151/152): librados2-19.0.0-3156.gd69f13d8.el7. 7.8 MB/s | 3.4 MB 00:00 2024-04-25T20:44:14.797 INFO:teuthology.orchestra.run.smithi139.stdout:(152/152): librbd1-19.0.0-3156.gd69f13d8.el7.x8 4.2 MB/s | 3.1 MB 00:00 2024-04-25T20:44:14.805 INFO:teuthology.orchestra.run.smithi139.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:44:14.805 INFO:teuthology.orchestra.run.smithi139.stdout:Total 11 MB/s | 222 MB 00:19 2024-04-25T20:44:15.665 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2024-04-25T20:44:15.666 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2024-04-25T20:44:15.684 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-04-25T20:44:15.684 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-04-25T20:44:15.914 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction check 2024-04-25T20:44:16.056 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction check succeeded. 2024-04-25T20:44:16.056 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction test 2024-04-25T20:44:17.818 INFO:teuthology.orchestra.run.smithi139.stdout:Transaction test succeeded. 2024-04-25T20:44:17.818 INFO:teuthology.orchestra.run.smithi139.stdout:Running transaction 2024-04-25T20:44:21.659 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-04-25T20:44:21.710 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2024-04-25T20:44:21.786 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:44:21.797 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:44:21.835 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:44:21.889 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:44:21.923 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:44:21.949 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:44:21.991 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:44:22.005 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:44:22.058 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:44:22.117 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:44:22.150 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:44:22.194 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:44:22.326 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:44:22.433 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:44:22.521 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:44:22.563 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:44:22.601 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:44:22.626 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:44:22.692 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:44:22.730 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:44:22.747 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:44:22.786 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:44:22.877 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:44:22.877 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:44:22.915 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:44:22.926 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:44:22.955 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:44:23.058 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:44:23.061 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:44:23.080 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:44:23.111 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:44:23.139 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:44:23.143 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:44:23.164 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:44:23.185 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:44:23.234 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:44:23.257 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:44:23.320 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:44:23.341 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:44:23.356 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:44:23.363 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:44:23.389 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:44:23.496 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:44:23.558 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:44:23.584 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:44:23.591 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:44:23.622 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:44:23.656 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:44:23.702 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:44:23.721 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:44:23.764 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:44:23.794 INFO:teuthology.orchestra.run.smithi139.stdout: Preparing : 1/1 2024-04-25T20:44:23.796 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:44:23.807 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:44:23.832 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:44:23.839 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:44:23.868 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:44:23.868 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:44:23.884 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T20:44:23.922 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T20:44:23.935 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:44:23.940 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:44:23.974 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:44:23.989 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:44:24.018 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:44:24.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:44:24.050 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:44:24.055 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:44:24.060 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T20:44:24.078 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:44:24.131 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:44:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:44:24.166 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:44:24.209 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:44:24.225 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:44:24.238 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:44:24.244 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T20:44:24.256 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:44:24.269 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T20:44:24.291 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:44:24.307 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:44:24.312 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T20:44:24.353 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:44:24.368 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:44:24.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:44:24.427 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:44:24.470 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:44:24.499 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:44:24.558 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:44:24.569 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:44:24.613 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:44:24.622 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:44:24.641 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T20:44:24.696 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:44:24.753 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:44:24.828 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:44:24.874 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:44:24.881 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:44:24.904 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:44:24.996 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:44:25.025 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:44:25.134 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:44:25.173 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T20:44:25.217 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:44:25.221 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T20:44:25.246 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:44:25.249 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:44:25.280 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:44:25.319 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:44:25.345 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:44:25.360 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:44:25.367 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:44:25.399 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:44:25.413 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 14/156 2024-04-25T20:44:25.435 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:44:25.455 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T20:44:25.494 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:44:25.512 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:44:25.523 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:44:25.563 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:44:25.566 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:44:25.604 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:44:25.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:44:25.643 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:44:25.645 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:44:25.676 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T20:44:25.690 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:44:25.722 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:44:25.758 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:44:25.784 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T20:44:25.805 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:44:25.814 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:44:25.841 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:44:25.879 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T20:44:25.882 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:44:25.884 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:44:25.917 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T20:44:25.926 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:44:25.937 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:44:25.961 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T20:44:26.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:44:26.034 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T20:44:26.040 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:44:26.062 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:44:26.077 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T20:44:26.077 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:44:26.115 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:44:26.120 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T20:44:26.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:44:26.145 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:44:26.152 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T20:44:26.177 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:44:26.180 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T20:44:26.186 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:44:26.219 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:44:26.251 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T20:44:26.262 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:44:26.281 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:44:26.303 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T20:44:26.320 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:44:26.333 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T20:44:26.360 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:44:26.376 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T20:44:26.385 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:44:26.397 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:44:26.410 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:44:26.450 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:44:26.470 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T20:44:26.480 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:44:26.492 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:44:26.512 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 31/156 2024-04-25T20:44:26.522 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:44:26.555 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 32/156 2024-04-25T20:44:26.584 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 33/156 2024-04-25T20:44:26.617 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:44:26.619 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:44:26.650 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:44:26.654 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T20:44:26.672 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:44:26.678 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:44:26.699 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T20:44:26.712 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:44:26.712 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:44:26.738 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:44:26.751 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:44:26.762 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:44:26.774 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T20:44:26.783 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:44:26.813 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:44:26.837 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:44:26.847 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:44:26.881 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:44:26.909 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T20:44:26.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:44:26.957 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:44:26.965 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:44:26.968 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T20:44:27.042 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:44:27.095 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:44:27.206 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:44:27.320 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:44:27.344 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:44:27.392 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:44:27.432 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:44:27.480 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:44:27.492 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T20:44:27.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:44:27.520 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:44:27.553 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:44:27.558 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:44:27.571 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T20:44:27.592 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:44:27.600 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T20:44:27.625 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:44:27.643 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T20:44:27.667 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:44:27.681 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 43/156 2024-04-25T20:44:27.702 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:44:27.723 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T20:44:27.789 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:44:27.840 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:44:27.871 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:44:27.878 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T20:44:27.926 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T20:44:27.926 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:44:27.981 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T20:44:28.024 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:44:28.053 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T20:44:28.081 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:44:28.099 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T20:44:28.121 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T20:44:28.168 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T20:44:28.203 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T20:44:28.247 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T20:44:28.266 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:44:28.288 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T20:44:28.309 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:44:28.342 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:44:28.428 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T20:44:28.442 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:44:28.477 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T20:44:28.500 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:44:28.507 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T20:44:28.548 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T20:44:28.586 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:44:28.627 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:44:28.642 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T20:44:28.682 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T20:44:28.722 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T20:44:28.747 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T20:44:28.772 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T20:44:28.798 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:44:28.813 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:44:28.842 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:44:28.844 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:44:28.846 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T20:44:28.878 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:44:28.919 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:44:28.978 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:44:28.983 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T20:44:29.021 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T20:44:29.044 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:44:29.049 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T20:44:29.083 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T20:44:29.122 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T20:44:29.133 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:44:29.162 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T20:44:29.180 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:44:29.192 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T20:44:29.227 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T20:44:29.260 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T20:44:29.301 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T20:44:29.344 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T20:44:29.358 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:44:29.386 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:44:29.431 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T20:44:29.463 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:44:29.475 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T20:44:29.505 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T20:44:29.544 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T20:44:29.559 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:44:29.643 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T20:44:29.681 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:44:29.690 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T20:44:30.104 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:44:30.217 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:44:30.411 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T20:44:30.462 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T20:44:30.496 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T20:44:30.596 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T20:44:30.654 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T20:44:30.743 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T20:44:30.781 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T20:44:30.889 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:44:30.959 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T20:44:30.962 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:44:30.987 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T20:44:30.989 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:44:31.022 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:44:31.065 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T20:44:31.334 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:44:31.377 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:44:31.430 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:44:31.468 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:44:31.499 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:44:31.523 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:44:31.535 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:44:31.551 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:44:31.583 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:44:31.712 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T20:44:31.836 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T20:44:31.895 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:44:31.938 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:44:32.002 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:44:32.030 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:44:32.031 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:44:32.059 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:44:32.095 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:44:32.101 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:44:32.123 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:44:32.566 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:44:32.591 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:44:32.661 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:44:32.684 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:44:33.075 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T20:44:33.157 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T20:44:33.185 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T20:44:33.218 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T20:44:33.529 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T20:44:33.564 INFO:teuthology.orchestra.run.smithi139.stdout: Upgrading : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:44:33.655 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 99/156 2024-04-25T20:44:33.685 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 100/156 2024-04-25T20:44:33.721 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T20:44:34.198 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T20:44:34.225 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:44:34.296 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 103/156 2024-04-25T20:44:34.318 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 104/156 2024-04-25T20:44:34.394 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:34.445 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:34.930 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:34.965 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:34.985 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:35.032 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:44:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:44:35.116 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:44:35.116 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:44:35.116 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:35.141 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:44:35.504 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:35.552 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:44:35.634 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:44:35.635 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:44:35.635 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:44:35.635 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:35.653 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:44:36.650 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:36.681 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:37.225 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 105/156 2024-04-25T20:44:37.259 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:44:37.349 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 106/156 2024-04-25T20:44:37.349 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T20:44:37.349 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T20:44:37.349 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:37.385 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:44:46.694 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:44:46.694 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2024-04-25T20:44:46.694 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2024-04-25T20:44:46.694 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2024-04-25T20:44:46.694 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2024-04-25T20:44:46.694 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2024-04-25T20:44:46.695 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /root 2024-04-25T20:44:46.695 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2024-04-25T20:44:46.695 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:46.870 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:44:46.885 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:44:46.885 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:46.885 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:44:46.885 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:47.219 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-04-25T20:44:47.220 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:44:47.411 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:44:47.412 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:47.412 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:44:47.412 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:48.331 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:44:48.356 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:44:48.356 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:48.357 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:44:48.357 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:44:48.357 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:44:48.357 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:48.590 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:44:48.612 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:44:48.613 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:48.613 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:44:48.613 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:44:48.613 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:44:48.614 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:48.892 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:44:48.917 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:44:48.917 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:48.917 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:44:48.917 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:44:48.917 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:44:48.918 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:48.927 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 107/156 2024-04-25T20:44:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /sys 2024-04-25T20:44:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /proc 2024-04-25T20:44:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /mnt 2024-04-25T20:44:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /var/tmp 2024-04-25T20:44:48.927 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /home 2024-04-25T20:44:48.928 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /root 2024-04-25T20:44:48.928 INFO:teuthology.orchestra.run.smithi139.stdout:skipping the directory /tmp 2024-04-25T20:44:48.928 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:49.049 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:44:49.071 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:44:49.071 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:49.071 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:44:49.071 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:44:49.071 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:44:49.071 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:49.113 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:44:49.142 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:44:49.152 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 108/156 2024-04-25T20:44:49.152 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:49.152 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T20:44:49.152 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:49.165 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:44:49.165 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:49.165 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:44:49.165 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:44:49.165 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:44:49.165 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:49.180 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:44:49.236 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:44:49.292 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:44:49.477 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:44:49.533 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:44:49.601 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:44:49.610 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:44:49.624 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:44:49.624 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:49.624 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:44:49.624 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:44:49.624 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:44:49.624 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:49.699 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:44:49.723 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:44:49.771 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:44:49.826 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:44:49.839 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:44:49.875 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:44:49.907 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:44:49.934 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:44:50.011 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:44:50.068 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:44:50.137 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:44:50.225 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:44:50.356 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:44:50.379 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:44:50.393 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:44:50.408 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:44:50.425 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:44:50.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:44:50.453 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:44:50.502 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:44:50.531 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:44:50.632 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:44:50.657 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 109/156 2024-04-25T20:44:50.658 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:50.658 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T20:44:50.658 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:44:50.658 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T20:44:50.658 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:50.882 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:44:50.905 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:44:50.905 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 110/156 2024-04-25T20:44:50.906 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:50.906 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T20:44:50.906 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:44:50.906 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T20:44:50.906 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:50.943 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:44:50.975 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:44:51.036 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:44:51.066 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:44:51.341 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:44:51.364 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 111/156 2024-04-25T20:44:51.364 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:51.364 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T20:44:51.364 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:44:51.364 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T20:44:51.364 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:51.472 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T20:44:51.536 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T20:44:51.563 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:44:51.592 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T20:44:51.599 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:44:51.778 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T20:44:51.826 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T20:44:51.903 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T20:44:51.991 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:44:52.107 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:44:52.132 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:44:52.168 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:44:52.175 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T20:44:52.207 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T20:44:52.243 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T20:44:52.557 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:44:52.583 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:44:52.684 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:44:52.686 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:44:52.716 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 123/156 2024-04-25T20:44:52.749 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 124/156 2024-04-25T20:44:52.811 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 125/156 2024-04-25T20:44:52.847 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:44:52.848 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:44:52.877 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:44:52.900 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:44:52.900 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:52.900 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:44:52.900 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:44:52.901 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:44:52.901 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:52.945 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:44:52.955 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:44:53.053 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:44:53.101 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:44:53.129 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:44:53.160 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:44:53.262 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:44:53.423 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:44:53.462 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:44:53.485 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:44:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:44:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:44:53.485 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:44:53.485 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:53.537 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:44:53.548 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:44:53.645 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:44:53.693 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:44:53.892 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 126/156 2024-04-25T20:44:53.925 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:44:54.020 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:44:54.055 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:44:54.077 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:44:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:44:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:44:54.078 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:44:54.078 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:54.122 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:44:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:44:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.144 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:44:54.144 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:54.190 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:44:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:44:54.212 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.212 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:44:54.212 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:44:54.212 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:44:54.212 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:54.515 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:44:54.537 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:44:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:44:54.537 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:44:54.538 INFO:teuthology.orchestra.run.smithi097.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:44:54.538 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:44:54.629 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:44:54.656 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:44:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:44:54.682 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.682 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:44:54.682 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:44:54.682 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:44:54.682 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:54.749 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:44:54.770 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:44:54.771 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.771 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:44:54.771 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:54.816 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:44:54.838 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:44:54.838 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:54.839 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:44:54.839 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:44:54.839 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:44:54.839 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:54.891 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 127/156 2024-04-25T20:44:54.916 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:44:55.018 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 128/156 2024-04-25T20:44:55.150 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:44:55.173 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:44:55.173 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:55.173 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:44:55.173 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:44:55.173 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:44:55.173 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:44:55.179 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 129/156 2024-04-25T20:44:55.210 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:44:55.233 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 130/156 2024-04-25T20:44:55.233 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:55.234 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T20:44:55.234 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:44:55.234 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T20:44:55.234 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:55.277 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:44:55.289 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 131/156 2024-04-25T20:44:55.387 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 132/156 2024-04-25T20:44:55.434 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 133/156 2024-04-25T20:44:56.362 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T20:44:56.396 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:44:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 135/156 2024-04-25T20:44:56.419 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:56.419 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T20:44:56.419 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:44:56.419 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T20:44:56.419 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:56.464 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:44:56.486 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 136/156 2024-04-25T20:44:56.486 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:56.486 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T20:44:56.486 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:56.531 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:44:56.553 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 137/156 2024-04-25T20:44:56.554 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:56.554 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T20:44:56.554 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:44:56.554 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T20:44:56.554 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:56.865 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:44:56.887 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 138/156 2024-04-25T20:44:56.888 INFO:teuthology.orchestra.run.smithi139.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T20:44:56.888 INFO:teuthology.orchestra.run.smithi139.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T20:44:56.888 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:44:56.888 INFO:teuthology.orchestra.run.smithi139.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T20:44:56.888 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:44:59.102 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:44:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:44:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:44:59.216 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:44:59.244 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:44:59.427 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:44:59.465 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:44:59.495 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:44:59.770 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:44:59.810 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:44:59.851 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:44:59.884 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:44:59.911 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:45:00.099 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:45:00.114 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:00.148 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:45:00.150 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:00.170 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:00.178 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:45:00.725 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:00.768 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:00.788 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:01.003 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:45:01.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:45:01.138 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:45:01.219 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:45:01.251 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:45:01.251 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:45:01.335 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:45:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:45:01.368 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:45:01.455 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:45:01.501 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 139/156 2024-04-25T20:45:01.527 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:01.543 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 140/156 2024-04-25T20:45:01.600 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 141/156 2024-04-25T20:45:01.610 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:45:01.632 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T20:45:01.660 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T20:45:01.697 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:45:01.738 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:45:01.818 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:45:01.844 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 144/156 2024-04-25T20:45:01.859 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:45:01.860 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:45:01.888 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T20:45:01.927 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 146/156 2024-04-25T20:45:01.952 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:45:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:45:01.985 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:45:02.069 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:45:02.132 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:02.541 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:02.582 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:02.603 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T20:45:03.436 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 148/156 2024-04-25T20:45:03.528 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T20:45:03.569 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T20:45:03.656 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T20:45:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T20:45:03.699 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:45:03.791 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T20:45:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:45:03.824 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:45:03.916 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T20:45:03.981 INFO:teuthology.orchestra.run.smithi139.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:07.197 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:07.197 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:45:07.197 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:45:07.198 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:45:07.199 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:45:07.200 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:45:07.200 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:45:07.211 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:45:07.211 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:45:07.211 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:45:07.211 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:45:07.212 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:45:07.213 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:45:07.214 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:45:07.215 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:45:07.216 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:45:07.217 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:45:07.218 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:45:07.219 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:45:07.220 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:45:07.221 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:45:08.099 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:45:08.100 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:45:08.101 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:45:08.102 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:45:08.103 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:45:08.104 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:45:08.105 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:45:08.106 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:45:08.107 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:45:08.108 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:45:08.109 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:45:08.110 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:45:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:45:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:45:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:45:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:45:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:45:08.111 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:45:09.280 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:09.280 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:45:09.280 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2024-04-25T20:45:09.280 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.281 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.282 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:45:09.283 INFO:teuthology.orchestra.run.smithi097.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:45:09.284 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:45:09.285 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:45:09.286 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:45:09.287 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:45:09.288 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:45:09.289 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:45:09.290 INFO:teuthology.orchestra.run.smithi097.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:45:09.291 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout: 2024-04-25T20:45:09.292 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 1/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 3/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 4/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3156.gd69f1 5/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 6/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 7/156 2024-04-25T20:45:09.539 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 8/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 9/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 10/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 11/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 12/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 13/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 14/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_ 15/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 16/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 17/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_6 18/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86 19/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 20/156 2024-04-25T20:45:09.540 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el 21/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7. 22/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_6 23/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 24/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 25/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 26/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 27/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 28/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 29/156 2024-04-25T20:45:09.541 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8. 30/156 2024-04-25T20:45:09.552 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noa 31/156 2024-04-25T20:45:09.552 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.n 32/156 2024-04-25T20:45:09.552 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69 33/156 2024-04-25T20:45:09.552 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el 34/156 2024-04-25T20:45:09.552 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 35/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.e 36/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 37/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 38/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 39/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T20:45:09.553 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T20:45:09.554 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T20:45:09.555 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T20:45:09.556 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T20:45:09.557 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T20:45:09.558 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T20:45:09.559 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T20:45:09.560 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 149/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T20:45:09.561 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 151/156 2024-04-25T20:45:09.562 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T20:45:09.562 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T20:45:09.562 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T20:45:09.562 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T20:45:09.703 DEBUG:teuthology.parallel:result is None 2024-04-25T20:45:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:45:10.198 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-04-25T20:45:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.198 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.199 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.200 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:45:10.201 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:45:10.202 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:45:10.203 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:45:10.204 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:45:10.205 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:45:10.206 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:45:10.207 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:45:10.208 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:10.209 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:45:10.210 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-04-25T20:45:10.576 DEBUG:teuthology.parallel:result is None 2024-04-25T20:45:11.638 INFO:teuthology.orchestra.run.smithi139.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout:Upgraded: 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: librados2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: librbd1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout:Installed: 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T20:45:11.639 INFO:teuthology.orchestra.run.smithi139.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-base-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-grafana-dashboards-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-immutable-object-cache-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mds-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-dashboard-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-modules-core-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.640 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mgr-rook-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-mon-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-osd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-prometheus-alerts-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-radosgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-selinux-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-test-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: ceph-volume-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: cephadm-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-shell-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: cephfs-top-2:19.0.0-3156.gd69f13d8.el7.noarch 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T20:45:11.641 INFO:teuthology.orchestra.run.smithi139.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T20:45:11.642 INFO:teuthology.orchestra.run.smithi139.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libcephfs2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libcephsqlite-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: librados-devel-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libradosstriper1-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: librgw2-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T20:45:11.643 INFO:teuthology.orchestra.run.smithi139.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T20:45:11.644 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-argparse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ceph-common-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cephfs-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T20:45:11.645 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T20:45:11.646 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T20:45:11.647 INFO:teuthology.orchestra.run.smithi139.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T20:45:11.648 INFO:teuthology.orchestra.run.smithi139.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rados-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T20:45:11.649 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rgw-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T20:45:11.650 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-fuse-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-mirror-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: rbd-nbd-2:19.0.0-3156.gd69f13d8.el7.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:45:11.651 INFO:teuthology.orchestra.run.smithi139.stdout:Complete! 2024-04-25T20:45:12.047 DEBUG:teuthology.parallel:result is None 2024-04-25T20:45:12.048 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:12.195 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:45:12.217 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:45:12.217 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:45:12.218 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:45:12.220 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:12.441 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:45:12.463 INFO:teuthology.orchestra.run.smithi097.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:45:12.463 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:45:12.463 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:45:12.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:12.604 DEBUG:teuthology.orchestra.run.smithi139:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T20:45:12.625 INFO:teuthology.orchestra.run.smithi139.stdout:19.0.0-3156.gd69f13d8.el7 2024-04-25T20:45:12.625 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3156.gd69f13d8.el7 2024-04-25T20:45:12.625 INFO:teuthology.task.install:The correct ceph version 19.0.0-3156.gd69f13d8 is installed. 2024-04-25T20:45:12.627 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T20:45:12.627 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:45:12.627 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:45:12.653 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:45:12.653 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:45:12.681 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:45:12.681 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T20:45:12.706 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T20:45:12.706 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:45:12.707 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:45:12.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:45:12.794 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:45:12.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:45:12.817 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:45:12.883 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:45:12.883 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T20:45:12.906 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T20:45:12.972 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T20:45:12.972 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:45:12.973 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:45:12.997 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:45:13.059 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:45:13.060 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:45:13.083 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:45:13.146 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:45:13.146 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T20:45:13.170 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T20:45:13.233 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T20:45:13.233 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:45:13.233 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:45:13.256 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:45:13.320 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:45:13.320 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:45:13.343 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:45:13.406 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:45:13.406 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T20:45:13.429 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T20:45:13.493 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-25T20:45:13.593 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T20:45:13.593 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:13.594 INFO:tasks.cephadm:Cluster fsid is b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:45:13.594 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-25T20:45:13.594 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.96', 'mon.b': '172.21.15.97', 'mon.c': '172.21.15.139'} 2024-04-25T20:45:13.594 INFO:tasks.cephadm:First mon is mon.a on smithi096 2024-04-25T20:45:13.594 INFO:tasks.cephadm:First mgr is x 2024-04-25T20:45:13.594 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-25T20:45:13.594 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-04-25T20:45:13.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2024-04-25T20:45:13.643 DEBUG:teuthology.orchestra.run.smithi139:> sudo hostname $(hostname -s) 2024-04-25T20:45:13.666 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-25T20:45:13.667 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:13.811 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'd69f13d827952bb07bb9b7973bca14f97d764d6f', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78795/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3156-gd69f13d8', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3156.gd69f13d8'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-22 12:00:06.590653', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-25T20:45:13.956 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 d69f13d827952bb07bb9b7973bca14f97d764d6f from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:45:13.957 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm 2024-04-25T20:45:13.958 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:14.598 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:45 /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:14.599 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:15.480 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:45 /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:15.480 DEBUG:teuthology.orchestra.run.smithi139:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/d69f13d827952bb07bb9b7973bca14f97d764d6f/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:16.182 INFO:teuthology.orchestra.run.smithi139.stdout:-rw-r--r--. 1 ubuntu ubuntu 781541 Apr 25 20:45 /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:16.182 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:16.201 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:16.221 DEBUG:teuthology.orchestra.run.smithi139:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-25T20:45:16.257 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f on all hosts... 2024-04-25T20:45:16.257 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:45:16.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:45:16.263 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f pull 2024-04-25T20:45:17.858 INFO:teuthology.orchestra.run.smithi139.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:45:17.912 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:45:17.913 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:45:45.615 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T20:45:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:45:45.615 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:45:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2024-04-25T20:45:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:45:45.616 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2024-04-25T20:45:45.616 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T20:45:49.805 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:45:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:45:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:45:49.805 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-04-25T20:45:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:45:49.806 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T20:45:49.806 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_version": "ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev)", 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "image_id": "da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76", 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "repo_digests": [ 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4" 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout: ] 2024-04-25T20:45:52.702 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T20:45:52.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-04-25T20:45:52.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2024-04-25T20:45:52.794 DEBUG:teuthology.orchestra.run.smithi139:> sudo mkdir -p /etc/ceph 2024-04-25T20:45:52.821 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-04-25T20:45:52.849 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2024-04-25T20:45:52.878 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 777 /etc/ceph 2024-04-25T20:45:52.902 INFO:tasks.cephadm:Writing seed config... 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-25T20:45:52.904 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-25T20:45:52.905 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-25T20:45:52.906 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-25T20:45:52.907 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-25T20:45:52.908 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:45:52.908 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-25T20:45:52.926 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b713778a-0344-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-25T20:45:52.926 DEBUG:teuthology.orchestra.run.smithi096:mon.a> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a.service 2024-04-25T20:45:52.968 DEBUG:teuthology.orchestra.run.smithi096:mgr.x> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x.service 2024-04-25T20:45:53.012 INFO:tasks.cephadm:Bootstrapping... 2024-04-25T20:45:53.012 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f -v bootstrap --fsid b713778a-0344-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.96 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:45:53.205 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-04-25T20:45:53.205 INFO:teuthology.orchestra.run.smithi096.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f', '-v', 'bootstrap', '--fsid', 'b713778a-0344-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.96', '--skip-admin-label'] 2024-04-25T20:45:53.205 INFO:teuthology.orchestra.run.smithi096.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-25T20:45:53.206 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying podman|docker is present... 2024-04-25T20:45:53.231 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T20:45:53.231 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying lvm2 is present... 2024-04-25T20:45:53.231 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying time synchronization is in place... 2024-04-25T20:45:53.239 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T20:45:53.239 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T20:45:53.245 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T20:45:53.245 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T20:45:53.254 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-04-25T20:45:53.257 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-04-25T20:45:53.257 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-04-25T20:45:53.257 INFO:teuthology.orchestra.run.smithi096.stdout:Repeating the final host check... 2024-04-25T20:45:53.281 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-25T20:45:53.282 INFO:teuthology.orchestra.run.smithi096.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-25T20:45:53.282 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl is present 2024-04-25T20:45:53.282 INFO:teuthology.orchestra.run.smithi096.stdout:lvcreate is present 2024-04-25T20:45:53.288 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-25T20:45:53.288 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-25T20:45:53.294 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-25T20:45:53.294 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T20:45:53.300 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout enabled 2024-04-25T20:45:53.306 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout active 2024-04-25T20:45:53.306 INFO:teuthology.orchestra.run.smithi096.stdout:Unit chronyd.service is enabled and running 2024-04-25T20:45:53.306 INFO:teuthology.orchestra.run.smithi096.stdout:Host looks OK 2024-04-25T20:45:53.306 INFO:teuthology.orchestra.run.smithi096.stdout:Cluster fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:45:53.306 INFO:teuthology.orchestra.run.smithi096.stdout:Acquiring lock 140648501804192 on /run/cephadm/b713778a-0344-11ef-bc93-c7b262605968.lock 2024-04-25T20:45:53.307 INFO:teuthology.orchestra.run.smithi096.stdout:Lock 140648501804192 acquired on /run/cephadm/b713778a-0344-11ef-bc93-c7b262605968.lock 2024-04-25T20:45:53.307 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 3300 ... 2024-04-25T20:45:53.307 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying IP 172.21.15.96 port 6789 ... 2024-04-25T20:45:53.307 INFO:teuthology.orchestra.run.smithi096.stdout:Base mon IP(s) is [172.21.15.96:3300, 172.21.15.96:6789], mon addrv is [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-04-25T20:45:53.310 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.96 metric 100 2024-04-25T20:45:53.310 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.96 metric 100 2024-04-25T20:45:53.313 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-25T20:45:53.313 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-25T20:45:53.313 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-25T20:45:53.315 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-25T20:45:53.315 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-25T20:45:53.315 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T20:45:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-25T20:45:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout inet6 fe80::45d2:c701:9f98:9709/64 scope link noprefixroute 2024-04-25T20:45:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-25T20:45:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-04-25T20:45:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:Mon IP `172.21.15.96` is in CIDR network `172.21.0.0/20` 2024-04-25T20:45:53.316 INFO:teuthology.orchestra.run.smithi096.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-25T20:45:53.317 INFO:teuthology.orchestra.run.smithi096.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-25T20:45:53.318 INFO:teuthology.orchestra.run.smithi096.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:45:56.440 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stdout da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T20:45:56.440 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f... 2024-04-25T20:45:56.441 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Getting image source signatures 2024-04-25T20:45:56.441 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:7f03586bcf9c6a2b4d690c118c4aa5b771f9c6f252d75a20fb283dd07093a738 2024-04-25T20:45:56.441 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-25T20:45:56.441 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Copying config sha256:da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 2024-04-25T20:45:56.441 INFO:teuthology.orchestra.run.smithi096.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-25T20:46:05.897 INFO:teuthology.orchestra.run.smithi096.stdout:ceph: stdout ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T20:46:05.897 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph version: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev) 2024-04-25T20:46:05.897 INFO:teuthology.orchestra.run.smithi096.stdout:Extracting ceph user uid/gid from container image... 2024-04-25T20:46:07.592 INFO:teuthology.orchestra.run.smithi096.stdout:stat: stdout 167 167 2024-04-25T20:46:07.592 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial keys... 2024-04-25T20:46:08.610 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQAPwSpmVizCNRAADfCK8YH2s/hPUwURI41lhw== 2024-04-25T20:46:09.592 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQAQwSpm3JHLORAAgk9RYQe1nhytPR+hlepqYg== 2024-04-25T20:46:10.548 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph-authtool: stdout AQARwSpmxfa+NxAA8ZtBbE+oSf4e+ktO3GiQFA== 2024-04-25T20:46:10.549 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial monmap... 2024-04-25T20:46:11.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T20:46:11.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-25T20:46:11.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:11.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T20:46:11.481 INFO:teuthology.orchestra.run.smithi096.stdout:monmaptool for a [v2:172.21.15.96:3300,v1:172.21.15.96:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-25T20:46:11.481 INFO:teuthology.orchestra.run.smithi096.stdout:setting min_mon_release = quincy 2024-04-25T20:46:11.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: set fsid to b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:11.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-25T20:46:11.481 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:46:11.481 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mon... 2024-04-25T20:46:13.004 INFO:teuthology.orchestra.run.smithi096.stdout:create mon.a on 2024-04-25T20:46:13.211 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-25T20:46:13.367 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-25T20:46:13.566 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b713778a-0344-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-b713778a-0344-11ef-bc93-c7b262605968.target. 2024-04-25T20:46:13.566 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b713778a-0344-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-b713778a-0344-11ef-bc93-c7b262605968.target. 2024-04-25T20:46:13.822 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a 2024-04-25T20:46:13.822 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-25T20:46:13.993 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b713778a-0344-11ef-bc93-c7b262605968.target.wants/ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b713778a-0344-11ef-bc93-c7b262605968@.service. 2024-04-25T20:46:14.550 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:46:14.550 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T20:46:14.557 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:46:14.557 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T20:46:14.558 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T20:46:14.558 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T20:46:14.558 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon to start... 2024-04-25T20:46:14.558 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mon... 2024-04-25T20:46:15.701 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:15 smithi096 ceph-mon[28918]: from='client.? 172.21.15.96:0/1792134275' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-25T20:46:15.708 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout cluster: 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout id: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout services: 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.611964s) 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout data: 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-25T20:46:15.709 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout pgs: 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4cd1c9700 1 Processor -- start 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4cd1c9700 1 -- start start 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4cd1c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c8106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4cd1c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8107330 con 0x7fc4c8106940 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c8106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c8106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36752/0 (socket says 172.21.15.96:36752) 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:14.999+0000 7fc4c6d9d700 1 -- 172.21.15.96:0/2817621845 learned_addr learned my addr 172.21.15.96:0/2817621845 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4c6d9d700 1 -- 172.21.15.96:0/2817621845 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c8109970 con 0x7fc4c8106940 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4c6d9d700 1 --2- 172.21.15.96:0/2817621845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c8106d60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098a0 tx=0x7fc4b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9680550baca01f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4c5d9b700 1 -- 172.21.15.96:0/2817621845 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8004160 con 0x7fc4c8106940 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4c5d9b700 1 -- 172.21.15.96:0/2817621845 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b8004300 con 0x7fc4c8106940 2024-04-25T20:46:15.710 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4c5d9b700 1 -- 172.21.15.96:0/2817621845 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b80036a0 con 0x7fc4c8106940 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/2817621845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 msgr2=0x7fc4c8106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4cd1c9700 1 --2- 172.21.15.96:0/2817621845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c8106d60 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc4b80098a0 tx=0x7fc4b8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/2817621845 shutdown_connections 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4cd1c9700 1 --2- 172.21.15.96:0/2817621845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c8106d60 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/2817621845 >> 172.21.15.96:0/2817621845 conn(0x7fc4c81022b0 msgr2=0x7fc4c81046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.000+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/2817621845 shutdown_connections 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.001+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/2817621845 wait complete. 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.001+0000 7fc4cd1c9700 1 Processor -- start 2024-04-25T20:46:15.711 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.001+0000 7fc4cd1c9700 1 -- start start 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4cd1c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4cd1c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8109ea0 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4c6d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c819bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36760/0 (socket says 172.21.15.96:36760) 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4c6d9d700 1 -- 172.21.15.96:0/1792134275 learned_addr learned my addr 172.21.15.96:0/1792134275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4c6d9d700 1 -- 172.21.15.96:0/1792134275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c819c1f0 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4c6d9d700 1 --2- 172.21.15.96:0/1792134275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c819bcb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc4c81076c0 tx=0x7fc4b8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8003db0 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4b8011630 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c819c480 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4b8011930 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.002+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c819f170 con 0x7fc4c8106940 2024-04-25T20:46:15.712 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.003+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc4b8011ad0 con 0x7fc4c8106940 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.003+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc4b801b450 con 0x7fc4c8106940 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.003+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c80640a0 con 0x7fc4c8106940 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.006+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fc4b8004260 con 0x7fc4c8106940 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.076+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc4c802e1b0 con 0x7fc4c8106940 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.077+0000 7fc4affff700 1 -- 172.21.15.96:0/1792134275 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fc4b8025480 con 0x7fc4c8106940 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 msgr2=0x7fc4c819bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 --2- 172.21.15.96:0/1792134275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c819bcb0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc4c81076c0 tx=0x7fc4b8004750 comp rx=0 tx=0).stop 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 shutdown_connections 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 --2- 172.21.15.96:0/1792134275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8106940 0x7fc4c819bcb0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 >> 172.21.15.96:0/1792134275 conn(0x7fc4c81022b0 msgr2=0x7fc4c8192990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 shutdown_connections 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:15.078+0000 7fc4cd1c9700 1 -- 172.21.15.96:0/1792134275 wait complete. 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:mon is available 2024-04-25T20:46:15.713 INFO:teuthology.orchestra.run.smithi096.stdout:Assimilating anything we can from ceph.conf... 2024-04-25T20:46:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:16 smithi096 ceph-mon[28918]: from='client.? 172.21.15.96:0/1392912511' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T20:46:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:16 smithi096 ceph-mon[28918]: from='client.? 172.21.15.96:0/1392912511' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-25T20:46:16.993 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:17.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T20:46:17.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.96:3300,v1:172.21.15.96:6789] 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T20:46:17.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5654585700 1 Processor -- start 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5654585700 1 -- start start 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5654585700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c1072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5654585700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f564c07ce50 con 0x7f564c106ed0 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5652321700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c1072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5652321700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c1072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36776/0 (socket says 172.21.15.96:36776) 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.116+0000 7f5652321700 1 -- 172.21.15.96:0/272750159 learned_addr learned my addr 172.21.15.96:0/272750159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5652321700 1 -- 172.21.15.96:0/272750159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c107830 con 0x7f564c106ed0 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5652321700 1 --2- 172.21.15.96:0/272750159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c1072f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f56480098a0 tx=0x7f5648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57ccb649229df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:17.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f565131f700 1 -- 172.21.15.96:0/272750159 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648004160 con 0x7f564c106ed0 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f565131f700 1 -- 172.21.15.96:0/272750159 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648004300 con 0x7f564c106ed0 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f565131f700 1 -- 172.21.15.96:0/272750159 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56480036a0 con 0x7f564c106ed0 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5654585700 1 -- 172.21.15.96:0/272750159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 msgr2=0x7f564c1072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5654585700 1 --2- 172.21.15.96:0/272750159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c1072f0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f56480098a0 tx=0x7f5648009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5654585700 1 -- 172.21.15.96:0/272750159 shutdown_connections 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5654585700 1 --2- 172.21.15.96:0/272750159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c1072f0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.117+0000 7f5654585700 1 -- 172.21.15.96:0/272750159 >> 172.21.15.96:0/272750159 conn(0x7f564c102ad0 msgr2=0x7f564c104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5654585700 1 -- 172.21.15.96:0/272750159 shutdown_connections 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5654585700 1 -- 172.21.15.96:0/272750159 wait complete. 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5654585700 1 Processor -- start 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5654585700 1 -- start start 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5654585700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:17.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5654585700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f564c107d60 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5652321700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5652321700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c19de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36782/0 (socket says 172.21.15.96:36782) 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.118+0000 7f5652321700 1 -- 172.21.15.96:0/1392912511 learned_addr learned my addr 172.21.15.96:0/1392912511 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f5652321700 1 -- 172.21.15.96:0/1392912511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c19e350 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f5652321700 1 --2- 172.21.15.96:0/1392912511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c19de10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5648000c00 tx=0x7f5648004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648003db0 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5648011630 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5648011950 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564c19e5e0 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564c1a12d0 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5648011af0 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.119+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f564801c450 con 0x7f564c106ed0 2024-04-25T20:46:17.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.120+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f564c0640a0 con 0x7f564c106ed0 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.122+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f5648004260 con 0x7f564c106ed0 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.189+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f564c1a14a0 con 0x7f564c106ed0 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.192+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+492 (secure 0 0 0) 0x7f564801b450 con 0x7f564c106ed0 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f56437fe700 1 -- 172.21.15.96:0/1392912511 <== mon.0 v2:172.21.15.96:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f564801b630 con 0x7f564c106ed0 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 msgr2=0x7f564c19de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 --2- 172.21.15.96:0/1392912511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c19de10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5648000c00 tx=0x7f5648004750 comp rx=0 tx=0).stop 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 shutdown_connections 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 --2- 172.21.15.96:0/1392912511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f564c106ed0 0x7f564c19de10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 >> 172.21.15.96:0/1392912511 conn(0x7f564c102ad0 msgr2=0x7f564c194450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 shutdown_connections 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:16.193+0000 7f5654585700 1 -- 172.21.15.96:0/1392912511 wait complete. 2024-04-25T20:46:17.005 INFO:teuthology.orchestra.run.smithi096.stdout:Generating new minimal ceph.conf... 2024-04-25T20:46:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:17 smithi096 ceph-mon[28918]: from='client.? 172.21.15.96:0/2308210734' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bd1bb2700 1 Processor -- start 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bd1bb2700 1 -- start start 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bd1bb2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc1070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bd1bb2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc1076c0 con 0x7f6bcc106cd0 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bcb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc1070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bcb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc1070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56550/0 (socket says 172.21.15.96:56550) 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bcb7fe700 1 -- 172.21.15.96:0/3384987959 learned_addr learned my addr 172.21.15.96:0/3384987959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.420+0000 7f6bcb7fe700 1 -- 172.21.15.96:0/3384987959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc107ed0 con 0x7f6bcc106cd0 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bcb7fe700 1 --2- 172.21.15.96:0/3384987959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc1070f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0098a0 tx=0x7f6bbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6f1ba2f345451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bca7fc700 1 -- 172.21.15.96:0/3384987959 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bbc004160 con 0x7f6bcc106cd0 2024-04-25T20:46:18.305 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bca7fc700 1 -- 172.21.15.96:0/3384987959 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6bbc004300 con 0x7f6bcc106cd0 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bca7fc700 1 -- 172.21.15.96:0/3384987959 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bbc011af0 con 0x7f6bcc106cd0 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/3384987959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 msgr2=0x7f6bcc1070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 --2- 172.21.15.96:0/3384987959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc1070f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc0098a0 tx=0x7f6bbc009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/3384987959 shutdown_connections 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 --2- 172.21.15.96:0/3384987959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc1070f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/3384987959 >> 172.21.15.96:0/3384987959 conn(0x7f6bcc1022b0 msgr2=0x7f6bcc1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/3384987959 shutdown_connections 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.421+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/3384987959 wait complete. 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bd1bb2700 1 Processor -- start 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bd1bb2700 1 -- start start 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bd1bb2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc19b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bd1bb2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6bcc10b100 con 0x7f6bcc106cd0 2024-04-25T20:46:18.306 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bcb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc19b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bcb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc19b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56554/0 (socket says 172.21.15.96:56554) 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.422+0000 7f6bcb7fe700 1 -- 172.21.15.96:0/2308210734 learned_addr learned my addr 172.21.15.96:0/2308210734 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bcb7fe700 1 -- 172.21.15.96:0/2308210734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bcc19ba00 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bcb7fe700 1 --2- 172.21.15.96:0/2308210734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc19b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009fb0 tx=0x7f6bbc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bbc0036a0 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bcc19bc90 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6bbc01a430 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6bbc019550 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bcc19e980 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6bbc019be0 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.423+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6bbc024730 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.424+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bcc10b280 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.426+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6bbc043e50 con 0x7f6bcc106cd0 2024-04-25T20:46:18.307 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.492+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6bcc02e140 con 0x7f6bcc106cd0 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.493+0000 7f6bc8ff9700 1 -- 172.21.15.96:0/2308210734 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f6bbc032030 con 0x7f6bcc106cd0 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.493+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 msgr2=0x7f6bcc19b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.493+0000 7f6bd1bb2700 1 --2- 172.21.15.96:0/2308210734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc19b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009fb0 tx=0x7f6bbc011940 comp rx=0 tx=0).stop 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.493+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 shutdown_connections 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.493+0000 7f6bd1bb2700 1 --2- 172.21.15.96:0/2308210734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6bcc106cd0 0x7f6bcc19b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.493+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 >> 172.21.15.96:0/2308210734 conn(0x7f6bcc1022b0 msgr2=0x7f6bcc1046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.494+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 shutdown_connections 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:17.494+0000 7f6bd1bb2700 1 -- 172.21.15.96:0/2308210734 wait complete. 2024-04-25T20:46:18.308 INFO:teuthology.orchestra.run.smithi096.stdout:Restarting the monitor... 2024-04-25T20:46:18.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:18 smithi096 systemd[1]: Stopping Ceph mon.a for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:46:18.934 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:18 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a[28914]: 2024-04-25T20:46:18.583+0000 7fe328f38700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T20:46:18.934 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:18 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a[28914]: 2024-04-25T20:46:18.583+0000 7fe328f38700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-25T20:46:18.934 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:18 smithi096 podman[29163]: 2024-04-25 20:46:18.675536457 +0000 UTC m=+0.188152324 container died 828be269c0c03b7a02fda9640b5e68226d222dc09f361844745a3b619eeb52dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-25T20:46:19.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 podman[29163]: 2024-04-25 20:46:19.033575486 +0000 UTC m=+0.546191339 container cleanup 828be269c0c03b7a02fda9640b5e68226d222dc09f361844745a3b619eeb52dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T20:46:19.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 bash[29163]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a 2024-04-25T20:46:19.710 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 podman[29175]: 2024-04-25 20:46:19.334701369 +0000 UTC m=+0.660395765 container remove 828be269c0c03b7a02fda9640b5e68226d222dc09f361844745a3b619eeb52dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T20:46:19.710 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 systemd[1]: ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-25T20:46:19.710 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 systemd[1]: Stopped Ceph mon.a for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:46:19.711 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 systemd[1]: Starting Ceph mon.a for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:46:19.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 podman[29270]: 2024-04-25 20:46:19.612753278 +0000 UTC m=+0.017981299 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f 2024-04-25T20:46:19.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 podman[29270]: 2024-04-25 20:46:19.718880441 +0000 UTC m=+0.124108435 container create 45ed8be0b8dbe0689e387f3bd278cc55f09bdc23d366e67456935626f479001d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS) 2024-04-25T20:46:19.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 podman[29270]: 2024-04-25 20:46:19.896016323 +0000 UTC m=+0.301244317 container init 45ed8be0b8dbe0689e387f3bd278cc55f09bdc23d366e67456935626f479001d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T20:46:19.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 podman[29270]: 2024-04-25 20:46:19.898972082 +0000 UTC m=+0.304200077 container start 45ed8be0b8dbe0689e387f3bd278cc55f09bdc23d366e67456935626f479001d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-25T20:46:19.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:46:19.963 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: pidfile_write: ignore empty --pid-file 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: load: jerasure load: lrc 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Git sha 0 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: DB SUMMARY 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: DB Session ID: E5E9WL8121W23DYH1URJ 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106032 ; 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:46:19.964 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.env: 0x55fe21b31cc0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.info_log: 0x55fe23625be0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.statistics: (nil) 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.use_fsync: 0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:46:19.965 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.db_log_dir: 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.wal_dir: 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:46:19.966 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.write_buffer_manager: 0x55fe235b90e0 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:46:19.967 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:46:19.968 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:46:19.968 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:46:19.968 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.unordered_write: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.row_cache: None 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.wal_filter: None 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.wal_compression: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:46:19.969 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:46:19.970 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_open_files: -1 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:46:19.971 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Compression algorithms supported: 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kZSTD supported: 0 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-25T20:46:19.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.merge_operator: 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_filter: None 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fe23625dc0) 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_type: 0 2024-04-25T20:46:19.973 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_index_type: 0 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_shortening: 1 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: checksum: 4 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: no_block_cache: 0 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache: 0x55fe235eb1f0 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_options: 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: capacity : 536870912 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_shard_bits : 4 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: strict_capacity_limit : 0 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_cache_compressed: (nil) 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: persistent_cache: (nil) 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size: 4096 2024-04-25T20:46:19.974 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_size_deviation: 10 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_restart_interval: 16 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: index_block_restart_interval: 1 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: metadata_block_size: 4096 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: partition_filters: 0 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: use_delta_encoding: 1 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: filter_policy: bloomfilter 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: whole_key_filtering: 1 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: verify_compression: 0 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:46:19.975 INFO:journalctl@ceph.mon.a.smithi096.stdout: format_version: 5 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_index_compression: 1 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout: block_align: 0 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout: max_auto_readahead_size: 262144 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout: prepopulate_block_cache: 0 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression: NoCompression 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.num_levels: 7 2024-04-25T20:46:19.976 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:46:19.977 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:46:19.978 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:46:19.979 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:46:19.980 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.ttl: 2592000 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:46:19.981 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-25T20:46:19.982 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7655237d-e418-4b53-825b-bd3a30025d72 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714077979928772, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714077979929857, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101084, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 330, "table_properties": {"data_size": 98973, "index_size": 303, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14405, "raw_average_key_size": 49, "raw_value_size": 91152, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 289, "num_filter_entries": 289, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714077979, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7655237d-e418-4b53-825b-bd3a30025d72", "db_session_id": "E5E9WL8121W23DYH1URJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714077979929916, "job": 1, "event": "recovery_finished"} 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55fe23700000 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: DB pointer 0x55fe236ea000 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** DB Stats ** 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:46:19.983 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: L0 2/0 100.58 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Sum 2/0 100.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 141.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T20:46:19.984 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** Compaction Stats [default] ** 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 141.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: Cumulative compaction: 0.00 GB write, 31.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:46:19.985 INFO:journalctl@ceph.mon.a.smithi096.stdout: Interval compaction: 0.00 GB write, 31.70 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache BinnedLRUCache@0x55fe235eb1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: starting mon.a rank 0 at public addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] at bind addrs [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???) e1 preinit fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).mds e1 new map 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).mds e1 print_map 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: e1 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: legacy client fscid: -1 2024-04-25T20:46:19.986 INFO:journalctl@ceph.mon.a.smithi096.stdout: 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout: No filesystems configured 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 bash[29270]: 45ed8be0b8dbe0689e387f3bd278cc55f09bdc23d366e67456935626f479001d 2024-04-25T20:46:19.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 systemd[1]: Started Ceph mon.a for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:46:19.987 INFO:teuthology.orchestra.run.smithi096.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-25T20:46:20.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-25T20:46:20.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: monmap epoch 1 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: last_changed 2024-04-25T20:46:10.879735+0000 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: min_mon_release 19 (squid) 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: election_strategy: 1 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: fsmap 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: osdmap e1: 0 total, 0 up, 0 in 2024-04-25T20:46:20.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:19 smithi096 ceph-mon[29288]: mgrmap e1: no daemons active 2024-04-25T20:46:21.184 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.568+0000 7fd03a9ef700 1 Processor -- start 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.568+0000 7fd03a9ef700 1 -- start start 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.568+0000 7fd03a9ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03407d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.568+0000 7fd03a9ef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd03407d820 con 0x7fd03407ee30 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.568+0000 7fd033fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03407d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd033fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03407d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56556/0 (socket says 172.21.15.96:56556) 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd033fff700 1 -- 172.21.15.96:0/3951201318 learned_addr learned my addr 172.21.15.96:0/3951201318 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd033fff700 1 -- 172.21.15.96:0/3951201318 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03407d9a0 con 0x7fd03407ee30 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd033fff700 1 --2- 172.21.15.96:0/3951201318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03407d2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0240098a0 tx=0x7fd024009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158a6b210ed53249 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd032ffd700 1 -- 172.21.15.96:0/3951201318 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd024004160 con 0x7fd03407ee30 2024-04-25T20:46:21.185 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd032ffd700 1 -- 172.21.15.96:0/3951201318 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd024004300 con 0x7fd03407ee30 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.569+0000 7fd032ffd700 1 -- 172.21.15.96:0/3951201318 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd024011af0 con 0x7fd03407ee30 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 -- 172.21.15.96:0/3951201318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 msgr2=0x7fd03407d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 --2- 172.21.15.96:0/3951201318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03407d2e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd0240098a0 tx=0x7fd024009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 -- 172.21.15.96:0/3951201318 shutdown_connections 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 --2- 172.21.15.96:0/3951201318 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03407d2e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 -- 172.21.15.96:0/3951201318 >> 172.21.15.96:0/3951201318 conn(0x7fd034103c30 msgr2=0x7fd034106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 -- 172.21.15.96:0/3951201318 shutdown_connections 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 -- 172.21.15.96:0/3951201318 wait complete. 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 Processor -- start 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.570+0000 7fd03a9ef700 1 -- start start 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd03a9ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd03a9ef700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410eb20 con 0x7fd03407ee30 2024-04-25T20:46:21.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd033fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd033fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03419e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56560/0 (socket says 172.21.15.96:56560) 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd033fff700 1 -- 172.21.15.96:0/1884558795 learned_addr learned my addr 172.21.15.96:0/1884558795 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd033fff700 1 -- 172.21.15.96:0/1884558795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03419ec60 con 0x7fd03407ee30 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd033fff700 1 --2- 172.21.15.96:0/1884558795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03419e720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd024009fb0 tx=0x7fd024011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd0240036a0 con 0x7fd03407ee30 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd02401a430 con 0x7fd03407ee30 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03419eef0 con 0x7fd03407ee30 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd024019550 con 0x7fd03407ee30 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.571+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a1be0 con 0x7fd03407ee30 2024-04-25T20:46:21.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.572+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd024019be0 con 0x7fd03407ee30 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.572+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd024024940 con 0x7fd03407ee30 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.572+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0340640a0 con 0x7fd03407ee30 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.575+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fd024043e50 con 0x7fd03407ee30 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.643+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7fd0341a1db0 con 0x7fd03407ee30 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.645+0000 7fd0317fa700 1 -- 172.21.15.96:0/1884558795 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7fd024032020 con 0x7fd03407ee30 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.645+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 msgr2=0x7fd03419e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.645+0000 7fd03a9ef700 1 --2- 172.21.15.96:0/1884558795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03419e720 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd024009fb0 tx=0x7fd024011940 comp rx=0 tx=0).stop 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.646+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 shutdown_connections 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.646+0000 7fd03a9ef700 1 --2- 172.21.15.96:0/1884558795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd03407ee30 0x7fd03419e720 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.646+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 >> 172.21.15.96:0/1884558795 conn(0x7fd034103c30 msgr2=0x7fd0341047a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.646+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 shutdown_connections 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:20.646+0000 7fd03a9ef700 1 -- 172.21.15.96:0/1884558795 wait complete. 2024-04-25T20:46:21.188 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-25T20:46:21.202 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:46:21.202 INFO:teuthology.orchestra.run.smithi096.stdout:Creating mgr... 2024-04-25T20:46:21.203 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-25T20:46:21.203 INFO:teuthology.orchestra.run.smithi096.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-25T20:46:21.447 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x 2024-04-25T20:46:21.447 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-25T20:46:21.593 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b713778a-0344-11ef-bc93-c7b262605968.target.wants/ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b713778a-0344-11ef-bc93-c7b262605968@.service. 2024-04-25T20:46:21.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:21 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1884558795' entity='client.admin' 2024-04-25T20:46:22.107 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 podman[29503]: 2024-04-25 20:46:22.043204601 +0000 UTC m=+0.330885973 container start 8c60889ad849595d42378f23a3798360f1dddb13916c86a305d8eeeb96660985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T20:46:22.122 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:46:22.123 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T20:46:22.130 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:46:22.130 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T20:46:22.130 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T20:46:22.131 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to enable service . firewalld.service is not available 2024-04-25T20:46:22.139 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:46:22.139 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T20:46:22.147 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:46:22.147 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T20:46:22.147 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T20:46:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-25T20:46:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr to start... 2024-04-25T20:46:22.148 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr... 2024-04-25T20:46:22.390 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 bash[29503]: 8c60889ad849595d42378f23a3798360f1dddb13916c86a305d8eeeb96660985 2024-04-25T20:46:22.390 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 systemd[1]: Started Ceph mgr.x for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:46:22.390 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:22.203+0000 7efede6b7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:46:22.390 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:22.388+0000 7efede6b7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:46:22.678 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:22.491+0000 7efede6b7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:46:22.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:22 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3901849028' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:46:23.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:22.811+0000 7efede6b7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:46:23.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:22.905+0000 7efede6b7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:46:23.469 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:23.469 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:46:23.469 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:46:23.470 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:23.471 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:46:23.472 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:46:23.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:46:23.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:46:14.467555+0000", 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:23.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.778+0000 7f76318ea700 1 Processor -- start 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.778+0000 7f76318ea700 1 -- start start 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.778+0000 7f76318ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c10b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.778+0000 7f76318ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f762c063d00 con 0x7f762c109110 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f762affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c10b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f762affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c10b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56588/0 (socket says 172.21.15.96:56588) 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f762affd700 1 -- 172.21.15.96:0/74622396 learned_addr learned my addr 172.21.15.96:0/74622396 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f762affd700 1 -- 172.21.15.96:0/74622396 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c10ba40 con 0x7f762c109110 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f762affd700 1 --2- 172.21.15.96:0/74622396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c10b500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76140098a0 tx=0x7f7614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde8f9ad1d1fcb3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f7629ffb700 1 -- 172.21.15.96:0/74622396 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7614004160 con 0x7f762c109110 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f7629ffb700 1 -- 172.21.15.96:0/74622396 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7614004300 con 0x7f762c109110 2024-04-25T20:46:23.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.779+0000 7f7629ffb700 1 -- 172.21.15.96:0/74622396 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7614011af0 con 0x7f762c109110 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 -- 172.21.15.96:0/74622396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 msgr2=0x7f762c10b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 --2- 172.21.15.96:0/74622396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c10b500 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f76140098a0 tx=0x7f7614009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 -- 172.21.15.96:0/74622396 shutdown_connections 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 --2- 172.21.15.96:0/74622396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c10b500 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 -- 172.21.15.96:0/74622396 >> 172.21.15.96:0/74622396 conn(0x7f762c102b80 msgr2=0x7f762c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 -- 172.21.15.96:0/74622396 shutdown_connections 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.780+0000 7f76318ea700 1 -- 172.21.15.96:0/74622396 wait complete. 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f76318ea700 1 Processor -- start 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f76318ea700 1 -- start start 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f76318ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f76318ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f762c10c940 con 0x7f762c109110 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f762affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f762affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56602/0 (socket says 172.21.15.96:56602) 2024-04-25T20:46:23.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f762affd700 1 -- 172.21.15.96:0/3901849028 learned_addr learned my addr 172.21.15.96:0/3901849028 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f762affd700 1 -- 172.21.15.96:0/3901849028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f762c19ca60 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.781+0000 7f762affd700 1 --2- 172.21.15.96:0/3901849028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c19c520 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7614011ac0 tx=0x7f7614011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7614003970 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7614003b10 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7614024610 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f762c19ccf0 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f762c19f9e0 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7614024bd0 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.782+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7614019700 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.783+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f762c0640a0 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.785+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f761401a9e0 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.858+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f762c19ff20 con 0x7f762c109110 2024-04-25T20:46:23.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.858+0000 7f76308e8700 1 -- 172.21.15.96:0/3901849028 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f761401a430 con 0x7f762c109110 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 msgr2=0x7f762c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 --2- 172.21.15.96:0/3901849028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c19c520 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7614011ac0 tx=0x7f7614011ba0 comp rx=0 tx=0).stop 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 shutdown_connections 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 --2- 172.21.15.96:0/3901849028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f762c109110 0x7f762c19c520 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 >> 172.21.15.96:0/3901849028 conn(0x7f762c102b80 msgr2=0x7f762c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 shutdown_connections 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:22.859+0000 7f76318ea700 1 -- 172.21.15.96:0/3901849028 wait complete. 2024-04-25T20:46:23.486 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (1/15)... 2024-04-25T20:46:24.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:24.118+0000 7efede6b7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:46:24.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:24.287+0000 7efede6b7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:46:24.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:24.456+0000 7efede6b7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:46:25.057 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:24.764+0000 7efede6b7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:46:25.057 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:24.856+0000 7efede6b7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:46:25.057 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:25 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:25.055+0000 7efede6b7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:46:25.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:25 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:25.248+0000 7efede6b7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:46:26.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3803305229' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:46:26.251 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:25 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:25.859+0000 7efede6b7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:46:26.593 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:26.294+0000 7efede6b7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:46:26.594 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:26.390+0000 7efede6b7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:46:26.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:46:26.626 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:46:26.626 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:46:26.634 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:46:26.635 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:46:14.467555+0000", 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f7093b06700 1 Processor -- start 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f7093b06700 1 -- start start 2024-04-25T20:46:26.636 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f7093b06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c10b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f7093b06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f708c07ce70 con 0x7f708c1090a0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f70918a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c10b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f70918a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c10b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56606/0 (socket says 172.21.15.96:56606) 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f70918a2700 1 -- 172.21.15.96:0/706783021 learned_addr learned my addr 172.21.15.96:0/706783021 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.906+0000 7f70918a2700 1 -- 172.21.15.96:0/706783021 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c10b9d0 con 0x7f708c1090a0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.907+0000 7f70918a2700 1 --2- 172.21.15.96:0/706783021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c10b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70880098a0 tx=0x7f7088009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b757fef186e80f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.907+0000 7f70908a0700 1 -- 172.21.15.96:0/706783021 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7088004160 con 0x7f708c1090a0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.907+0000 7f70908a0700 1 -- 172.21.15.96:0/706783021 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7088004300 con 0x7f708c1090a0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.907+0000 7f70908a0700 1 -- 172.21.15.96:0/706783021 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7088011af0 con 0x7f708c1090a0 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 -- 172.21.15.96:0/706783021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 msgr2=0x7f708c10b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 --2- 172.21.15.96:0/706783021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c10b490 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70880098a0 tx=0x7f7088009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 -- 172.21.15.96:0/706783021 shutdown_connections 2024-04-25T20:46:26.637 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 --2- 172.21.15.96:0/706783021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c10b490 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 -- 172.21.15.96:0/706783021 >> 172.21.15.96:0/706783021 conn(0x7f708c102af0 msgr2=0x7f708c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 -- 172.21.15.96:0/706783021 shutdown_connections 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 -- 172.21.15.96:0/706783021 wait complete. 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 Processor -- start 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.908+0000 7f7093b06700 1 -- start start 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f7093b06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f7093b06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f708c10c920 con 0x7f708c1090a0 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f70918a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f70918a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56612/0 (socket says 172.21.15.96:56612) 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f70918a2700 1 -- 172.21.15.96:0/3803305229 learned_addr learned my addr 172.21.15.96:0/3803305229 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f70918a2700 1 -- 172.21.15.96:0/3803305229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f708c19ca60 con 0x7f708c1090a0 2024-04-25T20:46:26.638 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f70918a2700 1 --2- 172.21.15.96:0/3803305229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c19c520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7088009fb0 tx=0x7f7088011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70880036a0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f708801a430 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.909+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f708c19ccf0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.910+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7088019550 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.910+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f708c19f9e0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.910+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f708801a5e0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.910+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f708802d780 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.911+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f708c0640a0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.913+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f7088003bd0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.985+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f708c19fbb0 con 0x7f708c1090a0 2024-04-25T20:46:26.639 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.986+0000 7f7082ffd700 1 -- 172.21.15.96:0/3803305229 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f7088019d90 con 0x7f708c1090a0 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 msgr2=0x7f708c19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 --2- 172.21.15.96:0/3803305229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c19c520 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7088009fb0 tx=0x7f7088011940 comp rx=0 tx=0).stop 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 shutdown_connections 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 --2- 172.21.15.96:0/3803305229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f708c1090a0 0x7f708c19c520 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 >> 172.21.15.96:0/3803305229 conn(0x7f708c102af0 msgr2=0x7f708c103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 shutdown_connections 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:25.987+0000 7f7093b06700 1 -- 172.21.15.96:0/3803305229 wait complete. 2024-04-25T20:46:26.640 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (2/15)... 2024-04-25T20:46:26.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:26.595+0000 7efede6b7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:46:27.447 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:27.110+0000 7efede6b7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:46:27.447 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:27.203+0000 7efede6b7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:46:27.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:27.446+0000 7efede6b7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:46:27.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:27.578+0000 7efede6b7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:46:27.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:27.671+0000 7efede6b7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:46:28.082 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:27.785+0000 7efede6b7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:46:28.082 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:28.080+0000 7efede6b7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:46:28.992 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:28.711+0000 7efede6b7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:46:28.992 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:28.935+0000 7efede6b7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:46:29.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: Activating manager daemon x 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: mgrmap e2: x(active, starting, since 0.0032214s) 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: Manager daemon x is now available 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:46:29.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' 2024-04-25T20:46:29.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:28 smithi096 ceph-mon[29288]: from='mgr.14100 172.21.15.96:0/1207140992' entity='mgr.x' 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:29.849 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:46:29.850 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:46:29.851 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:46:29.859 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:46:29.860 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:46:14.467555+0000", 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.051+0000 7fe6784b6700 1 Processor -- start 2024-04-25T20:46:29.861 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.051+0000 7fe6784b6700 1 -- start start 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.051+0000 7fe6784b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67010b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe6784b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe67007ceb0 con 0x7fe670109060 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe676252700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67010b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe676252700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67010b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47278/0 (socket says 172.21.15.96:47278) 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe676252700 1 -- 172.21.15.96:0/3721538284 learned_addr learned my addr 172.21.15.96:0/3721538284 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe676252700 1 -- 172.21.15.96:0/3721538284 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67010b990 con 0x7fe670109060 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe676252700 1 --2- 172.21.15.96:0/3721538284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67010b450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe6680098a0 tx=0x7fe668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa853032122d578f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe675250700 1 -- 172.21.15.96:0/3721538284 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe668004160 con 0x7fe670109060 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe675250700 1 -- 172.21.15.96:0/3721538284 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe668004300 con 0x7fe670109060 2024-04-25T20:46:29.862 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.052+0000 7fe675250700 1 -- 172.21.15.96:0/3721538284 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe668011af0 con 0x7fe670109060 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 -- 172.21.15.96:0/3721538284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 msgr2=0x7fe67010b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 --2- 172.21.15.96:0/3721538284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67010b450 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe6680098a0 tx=0x7fe668009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 -- 172.21.15.96:0/3721538284 shutdown_connections 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 --2- 172.21.15.96:0/3721538284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67010b450 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 -- 172.21.15.96:0/3721538284 >> 172.21.15.96:0/3721538284 conn(0x7fe670102af0 msgr2=0x7fe670104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 -- 172.21.15.96:0/3721538284 shutdown_connections 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.053+0000 7fe6784b6700 1 -- 172.21.15.96:0/3721538284 wait complete. 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe6784b6700 1 Processor -- start 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe6784b6700 1 -- start start 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe6784b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe6784b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe67010c8f0 con 0x7fe670109060 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe676252700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe676252700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67019c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47280/0 (socket says 172.21.15.96:47280) 2024-04-25T20:46:29.863 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe676252700 1 -- 172.21.15.96:0/1352625295 learned_addr learned my addr 172.21.15.96:0/1352625295 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe676252700 1 -- 172.21.15.96:0/1352625295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67019ca60 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.054+0000 7fe676252700 1 --2- 172.21.15.96:0/1352625295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67019c520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe668011ac0 tx=0x7fe668011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.055+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe668003970 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.055+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe668003b10 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.055+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67019ccf0 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.055+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe668024610 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.055+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67019f9e0 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.056+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6700640a0 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.056+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7fe66801a9e0 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.056+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe66804d470 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.058+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fe66800b3e0 con 0x7fe670109060 2024-04-25T20:46:29.864 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.130+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe67019fbb0 con 0x7fe670109060 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.131+0000 7fe6677fe700 1 -- 172.21.15.96:0/1352625295 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe67019fbb0 con 0x7fe670109060 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 msgr2=0x7fe67019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 --2- 172.21.15.96:0/1352625295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67019c520 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fe668011ac0 tx=0x7fe668011ba0 comp rx=0 tx=0).stop 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 shutdown_connections 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 --2- 172.21.15.96:0/1352625295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe670109060 0x7fe67019c520 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 >> 172.21.15.96:0/1352625295 conn(0x7fe670102af0 msgr2=0x7fe6701036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 shutdown_connections 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:29.132+0000 7fe6784b6700 1 -- 172.21.15.96:0/1352625295 wait complete. 2024-04-25T20:46:29.865 INFO:teuthology.orchestra.run.smithi096.stdout:mgr not available, waiting (3/15)... 2024-04-25T20:46:30.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:29 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1352625295' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:46:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:29 smithi096 ceph-mon[29288]: mgrmap e3: x(active, since 1.00599s) 2024-04-25T20:46:32.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:31 smithi096 ceph-mon[29288]: mgrmap e4: x(active, since 2s) 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "health": { 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-25T20:46:33.213 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 0 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "a" 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.214 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-25T20:46:33.215 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:33.215 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-25T20:46:33.215 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-25T20:46:33.215 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-25T20:46:33.223 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:33.224 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "restful" 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ], 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-25T20:46:33.225 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "modified": "2024-04-25T20:46:14.467555+0000", 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout }, 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.298+0000 7f452bdd5700 1 Processor -- start 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.298+0000 7f452bdd5700 1 -- start start 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.298+0000 7f452bdd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452407d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.298+0000 7f452bdd5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f452407d8d0 con 0x7f452407ee50 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4529b71700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452407d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4529b71700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452407d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47286/0 (socket says 172.21.15.96:47286) 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4529b71700 1 -- 172.21.15.96:0/523181935 learned_addr learned my addr 172.21.15.96:0/523181935 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4529b71700 1 -- 172.21.15.96:0/523181935 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452407da50 con 0x7f452407ee50 2024-04-25T20:46:33.226 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4529b71700 1 --2- 172.21.15.96:0/523181935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452407d300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9ccce1f22b0d172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4528b6f700 1 -- 172.21.15.96:0/523181935 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c004160 con 0x7f452407ee50 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4528b6f700 1 -- 172.21.15.96:0/523181935 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f451c004300 con 0x7f452407ee50 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.299+0000 7f4528b6f700 1 -- 172.21.15.96:0/523181935 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c011af0 con 0x7f452407ee50 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 -- 172.21.15.96:0/523181935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 msgr2=0x7f452407d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 --2- 172.21.15.96:0/523181935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452407d300 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f451c0098a0 tx=0x7f451c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 -- 172.21.15.96:0/523181935 shutdown_connections 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 --2- 172.21.15.96:0/523181935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452407d300 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 -- 172.21.15.96:0/523181935 >> 172.21.15.96:0/523181935 conn(0x7f4524103c40 msgr2=0x7f4524106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 -- 172.21.15.96:0/523181935 shutdown_connections 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 -- 172.21.15.96:0/523181935 wait complete. 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.300+0000 7f452bdd5700 1 Processor -- start 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f452bdd5700 1 -- start start 2024-04-25T20:46:33.227 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f452bdd5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f452bdd5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f452410eb50 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f4529b71700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f4529b71700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452419e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47302/0 (socket says 172.21.15.96:47302) 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f4529b71700 1 -- 172.21.15.96:0/3783539842 learned_addr learned my addr 172.21.15.96:0/3783539842 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f4529b71700 1 -- 172.21.15.96:0/3783539842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f452419ec70 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f4529b71700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452419e730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f451c009fb0 tx=0x7f451c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c0036a0 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f451c01a430 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f451c019550 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f452419ef00 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.301+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45241a1bf0 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.302+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f451c019800 con 0x7f452407ee50 2024-04-25T20:46:33.228 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.302+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45240640a0 con 0x7f452407ee50 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.302+0000 7f451affd700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7f4510039560 0x7f451003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.302+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f451c04d9b0 con 0x7f452407ee50 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.302+0000 7f4529370700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7f4510039560 0x7f451003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.303+0000 7f4529370700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7f4510039560 0x7f451003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4514006fd0 tx=0x7f4514006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.308+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f451c04d3d0 con 0x7f452407ee50 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.587+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f45241a1dc0 con 0x7f452407ee50 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.587+0000 7f451affd700 1 -- 172.21.15.96:0/3783539842 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f45241a1dc0 con 0x7f452407ee50 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7f4510039560 msgr2=0x7f451003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7f4510039560 0x7f451003ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4514006fd0 tx=0x7f4514006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 msgr2=0x7f452419e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452419e730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f451c009fb0 tx=0x7f451c011940 comp rx=0 tx=0).stop 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 shutdown_connections 2024-04-25T20:46:33.229 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7f4510039560 0x7f451003ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:33.230 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 --2- 172.21.15.96:0/3783539842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f452407ee50 0x7f452419e730 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:33.230 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 >> 172.21.15.96:0/3783539842 conn(0x7f4524103c40 msgr2=0x7f45241047d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:33.230 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 shutdown_connections 2024-04-25T20:46:33.230 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:32.589+0000 7f452bdd5700 1 -- 172.21.15.96:0/3783539842 wait complete. 2024-04-25T20:46:33.230 INFO:teuthology.orchestra.run.smithi096.stdout:mgr is available 2024-04-25T20:46:33.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:32 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3783539842' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-25T20:46:34.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:33 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2987572863' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-25T20:46:34.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:34.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [global] 2024-04-25T20:46:34.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout fsid = b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:34.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-25T20:46:34.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout [osd] 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-25T20:46:34.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe784976700 1 Processor -- start 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe784976700 1 -- start start 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe784976700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78010b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe784976700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe78007ce50 con 0x7fe7801090e0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78010b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78010b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47310/0 (socket says 172.21.15.96:47310) 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.591+0000 7fe77e59c700 1 -- 172.21.15.96:0/2502470940 learned_addr learned my addr 172.21.15.96:0/2502470940 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe77e59c700 1 -- 172.21.15.96:0/2502470940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78010ba10 con 0x7fe7801090e0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe77e59c700 1 --2- 172.21.15.96:0/2502470940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78010b4d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098a0 tx=0x7fe770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faae6899b5f4ddca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe77d59a700 1 -- 172.21.15.96:0/2502470940 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe770004160 con 0x7fe7801090e0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe77d59a700 1 -- 172.21.15.96:0/2502470940 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe770004300 con 0x7fe7801090e0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe77d59a700 1 -- 172.21.15.96:0/2502470940 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe770011af0 con 0x7fe7801090e0 2024-04-25T20:46:34.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe784976700 1 -- 172.21.15.96:0/2502470940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 msgr2=0x7fe78010b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.592+0000 7fe784976700 1 --2- 172.21.15.96:0/2502470940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78010b4d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe7700098a0 tx=0x7fe770009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 -- 172.21.15.96:0/2502470940 shutdown_connections 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 --2- 172.21.15.96:0/2502470940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78010b4d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 -- 172.21.15.96:0/2502470940 >> 172.21.15.96:0/2502470940 conn(0x7fe780102ad0 msgr2=0x7fe780104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 -- 172.21.15.96:0/2502470940 shutdown_connections 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 -- 172.21.15.96:0/2502470940 wait complete. 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 Processor -- start 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.593+0000 7fe784976700 1 -- start start 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe784976700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78019c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe784976700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe78010c910 con 0x7fe7801090e0 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78019c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe77e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78019c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47322/0 (socket says 172.21.15.96:47322) 2024-04-25T20:46:34.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe77e59c700 1 -- 172.21.15.96:0/2987572863 learned_addr learned my addr 172.21.15.96:0/2987572863 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe77e59c700 1 -- 172.21.15.96:0/2987572863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78019ca50 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe77e59c700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78019c510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe770009fb0 tx=0x7fe770011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7700036a0 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe77001a430 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78019cce0 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe770019550 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.594+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe78019f9d0 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.595+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe770019850 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.595+0000 7fe76f7fe700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe768039510 0x7fe76803b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.595+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe77004dab0 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.595+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7800640a0 con 0x7fe7801090e0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.595+0000 7fe77dd9b700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe768039510 0x7fe76803b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:34.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.596+0000 7fe77dd9b700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe768039510 0x7fe76803b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe774006fd0 tx=0x7fe774006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.600+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe77000f2e0 con 0x7fe7801090e0 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.795+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fe7801a10c0 con 0x7fe7801090e0 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.796+0000 7fe76f7fe700 1 -- 172.21.15.96:0/2987572863 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+436 (secure 0 0 0) 0x7fe770032020 con 0x7fe7801090e0 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe768039510 msgr2=0x7fe76803b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe768039510 0x7fe76803b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe774006fd0 tx=0x7fe774006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 msgr2=0x7fe78019c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78019c510 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fe770009fb0 tx=0x7fe770011940 comp rx=0 tx=0).stop 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 shutdown_connections 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe768039510 0x7fe76803b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 --2- 172.21.15.96:0/2987572863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7801090e0 0x7fe78019c510 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 >> 172.21.15.96:0/2987572863 conn(0x7fe780102ad0 msgr2=0x7fe780103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 shutdown_connections 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:33.799+0000 7fe784976700 1 -- 172.21.15.96:0/2987572863 wait complete. 2024-04-25T20:46:34.371 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling cephadm module... 2024-04-25T20:46:35.438 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1158499978' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-25T20:46:35.438 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: ignoring --setuser ceph since I am not root 2024-04-25T20:46:35.438 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: ignoring --setgroup ceph since I am not root 2024-04-25T20:46:35.438 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:35.268+0000 7f5ed75f4200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:46:35.688 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe90c05e700 1 Processor -- start 2024-04-25T20:46:35.688 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe90c05e700 1 -- start start 2024-04-25T20:46:35.688 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe90c05e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90407d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe90c05e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe90407d8f0 con 0x7fe90407eec0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe909dfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90407d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe909dfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90407d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47338/0 (socket says 172.21.15.96:47338) 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.786+0000 7fe909dfa700 1 -- 172.21.15.96:0/2414712778 learned_addr learned my addr 172.21.15.96:0/2414712778 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe909dfa700 1 -- 172.21.15.96:0/2414712778 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90407da70 con 0x7fe90407eec0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe909dfa700 1 --2- 172.21.15.96:0/2414712778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90407d320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe9000098a0 tx=0x7fe900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64a14d0fa14b5c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe908df8700 1 -- 172.21.15.96:0/2414712778 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe900004160 con 0x7fe90407eec0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe908df8700 1 -- 172.21.15.96:0/2414712778 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe900004300 con 0x7fe90407eec0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe908df8700 1 -- 172.21.15.96:0/2414712778 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe900011af0 con 0x7fe90407eec0 2024-04-25T20:46:35.689 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe90c05e700 1 -- 172.21.15.96:0/2414712778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 msgr2=0x7fe90407d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe90c05e700 1 --2- 172.21.15.96:0/2414712778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90407d320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe9000098a0 tx=0x7fe900009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe90c05e700 1 -- 172.21.15.96:0/2414712778 shutdown_connections 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe90c05e700 1 --2- 172.21.15.96:0/2414712778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90407d320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.787+0000 7fe90c05e700 1 -- 172.21.15.96:0/2414712778 >> 172.21.15.96:0/2414712778 conn(0x7fe904103be0 msgr2=0x7fe904106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe90c05e700 1 -- 172.21.15.96:0/2414712778 shutdown_connections 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe90c05e700 1 -- 172.21.15.96:0/2414712778 wait complete. 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe90c05e700 1 Processor -- start 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe90c05e700 1 -- start start 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe90c05e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe90c05e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe90410eb30 con 0x7fe90407eec0 2024-04-25T20:46:35.690 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe909dfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe909dfa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90419e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47348/0 (socket says 172.21.15.96:47348) 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.788+0000 7fe909dfa700 1 -- 172.21.15.96:0/1158499978 learned_addr learned my addr 172.21.15.96:0/1158499978 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe909dfa700 1 -- 172.21.15.96:0/1158499978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe90419eca0 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe909dfa700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90419e760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe900009fb0 tx=0x7fe900011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9000036a0 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe90001a430 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90419ef30 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe900019550 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.789+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9041a1c20 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.790+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7fe900019850 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.790+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9040640a0 con 0x7fe90407eec0 2024-04-25T20:46:35.691 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.790+0000 7fe8faffd700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe8f0039510 0x7fe8f003b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.790+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe90004da00 con 0x7fe90407eec0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.790+0000 7fe9095f9700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe8f0039510 0x7fe8f003b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.791+0000 7fe9095f9700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe8f0039510 0x7fe8f003b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006fd0 tx=0x7fe8f4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:34.795+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe90004d420 con 0x7fe90407eec0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.051+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fe9041a1df0 con 0x7fe90407eec0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.102+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fe9041a1df0 con 0x7fe90407eec0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.104+0000 7fe8faffd700 1 -- 172.21.15.96:0/1158499978 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fe90002d960 con 0x7fe90407eec0 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe8f0039510 msgr2=0x7fe8f003b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe8f0039510 0x7fe8f003b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe8f4006fd0 tx=0x7fe8f4006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 msgr2=0x7fe90419e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:35.692 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90419e760 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe900009fb0 tx=0x7fe900011940 comp rx=0 tx=0).stop 2024-04-25T20:46:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 shutdown_connections 2024-04-25T20:46:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fe8f0039510 0x7fe8f003b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 --2- 172.21.15.96:0/1158499978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe90407eec0 0x7fe90419e760 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 >> 172.21.15.96:0/1158499978 conn(0x7fe904103be0 msgr2=0x7fe904104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 shutdown_connections 2024-04-25T20:46:35.693 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:35.106+0000 7fe90c05e700 1 -- 172.21.15.96:0/1158499978 wait complete. 2024-04-25T20:46:35.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:35.457+0000 7f5ed75f4200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:46:35.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:35.559+0000 7f5ed75f4200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:46:36.066 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:35.874+0000 7f5ed75f4200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:46:36.066 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:35.960+0000 7f5ed75f4200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:46:36.372 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:36 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1158499978' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-25T20:46:36.372 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:36 smithi096 ceph-mon[29288]: mgrmap e5: x(active, since 6s) 2024-04-25T20:46:37.050 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26cb06700 1 Processor -- start 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26cb06700 1 -- start start 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26cb06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26810b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26cb06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd26807ce50 con 0x7fd2681090c0 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26810b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26810b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47368/0 (socket says 172.21.15.96:47368) 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.165+0000 7fd26659c700 1 -- 172.21.15.96:0/3852767275 learned_addr learned my addr 172.21.15.96:0/3852767275 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:37.051 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.166+0000 7fd26659c700 1 -- 172.21.15.96:0/3852767275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26810b9f0 con 0x7fd2681090c0 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.166+0000 7fd26659c700 1 --2- 172.21.15.96:0/3852767275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26810b4b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd2580098a0 tx=0x7fd258009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17fe8b2b30cbd757 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.166+0000 7fd26559a700 1 -- 172.21.15.96:0/3852767275 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd258004160 con 0x7fd2681090c0 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.166+0000 7fd26559a700 1 -- 172.21.15.96:0/3852767275 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd258004300 con 0x7fd2681090c0 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.166+0000 7fd26559a700 1 -- 172.21.15.96:0/3852767275 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd258011af0 con 0x7fd2681090c0 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 -- 172.21.15.96:0/3852767275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 msgr2=0x7fd26810b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 --2- 172.21.15.96:0/3852767275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26810b4b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd2580098a0 tx=0x7fd258009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 -- 172.21.15.96:0/3852767275 shutdown_connections 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 --2- 172.21.15.96:0/3852767275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26810b4b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 -- 172.21.15.96:0/3852767275 >> 172.21.15.96:0/3852767275 conn(0x7fd268102af0 msgr2=0x7fd268104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 -- 172.21.15.96:0/3852767275 shutdown_connections 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.167+0000 7fd26cb06700 1 -- 172.21.15.96:0/3852767275 wait complete. 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26cb06700 1 Processor -- start 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26cb06700 1 -- start start 2024-04-25T20:46:37.052 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26cb06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26819c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26cb06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd26810c8f0 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26819c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26819c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47374/0 (socket says 172.21.15.96:47374) 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26659c700 1 -- 172.21.15.96:0/1937114478 learned_addr learned my addr 172.21.15.96:0/1937114478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26659c700 1 -- 172.21.15.96:0/1937114478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26819ca60 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.168+0000 7fd26659c700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26819c520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd258011ac0 tx=0x7fd258011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.169+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd258003970 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.169+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd258003b10 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.169+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26819ccf0 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.169+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd258024610 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.169+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26819f9e0 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.170+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fd258024890 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.170+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2680640a0 con 0x7fd2681090c0 2024-04-25T20:46:37.053 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.170+0000 7fd2577fe700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 0x7fd25003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.170+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd25804db50 con 0x7fd2681090c0 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.170+0000 7fd265d9b700 1 -- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 msgr2=0x7fd25003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.170+0000 7fd265d9b700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 0x7fd25003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.179+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd25804d570 con 0x7fd2681090c0 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.371+0000 7fd265d9b700 1 -- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 msgr2=0x7fd25003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.371+0000 7fd265d9b700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 0x7fd25003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.472+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd26819fbb0 con 0x7fd2681090c0 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.472+0000 7fd2577fe700 1 -- 172.21.15.96:0/1937114478 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd26819fbb0 con 0x7fd2681090c0 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.474+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 msgr2=0x7fd25003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.474+0000 7fd26cb06700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 0x7fd25003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.474+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 msgr2=0x7fd26819c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.474+0000 7fd26cb06700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26819c520 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd258011ac0 tx=0x7fd258011ba0 comp rx=0 tx=0).stop 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.475+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 shutdown_connections 2024-04-25T20:46:37.054 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.475+0000 7fd26cb06700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fd2500395b0 0x7fd25003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:37.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.475+0000 7fd26cb06700 1 --2- 172.21.15.96:0/1937114478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2681090c0 0x7fd26819c520 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:37.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.475+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 >> 172.21.15.96:0/1937114478 conn(0x7fd268102af0 msgr2=0x7fd2681036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:37.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.475+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 shutdown_connections 2024-04-25T20:46:37.055 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:36.475+0000 7fd26cb06700 1 -- 172.21.15.96:0/1937114478 wait complete. 2024-04-25T20:46:37.055 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-04-25T20:46:37.055 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 5... 2024-04-25T20:46:37.304 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:37 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1937114478' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T20:46:37.304 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:37.177+0000 7f5ed75f4200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:46:37.713 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:37.353+0000 7f5ed75f4200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:46:37.713 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:37.529+0000 7f5ed75f4200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:46:37.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:37.819+0000 7f5ed75f4200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:46:38.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:37.911+0000 7f5ed75f4200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:46:38.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:38.111+0000 7f5ed75f4200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:46:38.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:38.303+0000 7f5ed75f4200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:46:39.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:38.918+0000 7f5ed75f4200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:46:39.639 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:39.337+0000 7f5ed75f4200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:46:39.639 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:39.439+0000 7f5ed75f4200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:46:39.640 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:39.639+0000 7f5ed75f4200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:46:40.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:40.178+0000 7f5ed75f4200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:46:40.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:40.269+0000 7f5ed75f4200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:46:40.857 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:40.511+0000 7f5ed75f4200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:46:40.857 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:40.645+0000 7f5ed75f4200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:46:40.857 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:40.740+0000 7f5ed75f4200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:46:40.857 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:40.856+0000 7f5ed75f4200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:46:41.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:41.158+0000 7f5ed75f4200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:46:42.021 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:41.754+0000 7f5ed75f4200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:46:42.021 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:46:41.968+0000 7f5ed75f4200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:46:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: Active manager daemon x restarted 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: Activating manager daemon x 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: osdmap e2: 0 total, 0 up, 0 in 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: mgrmap e6: x(active, starting, since 0.00351643s) 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: Manager daemon x is now available 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:46:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:46:42.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:42 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:46:43.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:43 smithi096 ceph-mon[29288]: Found migration_current of "None". Setting to last migration. 2024-04-25T20:46:43.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:43 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:43.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:43 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:43.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:43 smithi096 ceph-mon[29288]: mgrmap e7: x(active, since 1.00551s) 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b45a5700 1 Processor -- start 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b45a5700 1 -- start start 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b45a5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac10b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b45a5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac07ce50 con 0x7fb3ac109110 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b2341700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac10b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b2341700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac10b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56010/0 (socket says 172.21.15.96:56010) 2024-04-25T20:46:43.653 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.506+0000 7fb3b2341700 1 -- 172.21.15.96:0/2435935605 learned_addr learned my addr 172.21.15.96:0/2435935605 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.507+0000 7fb3b2341700 1 -- 172.21.15.96:0/2435935605 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac10ba40 con 0x7fb3ac109110 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.507+0000 7fb3b2341700 1 --2- 172.21.15.96:0/2435935605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac10b500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0098a0 tx=0x7fb39c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27021d95dbc0b906 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.507+0000 7fb3b133f700 1 -- 172.21.15.96:0/2435935605 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb39c004160 con 0x7fb3ac109110 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.507+0000 7fb3b133f700 1 -- 172.21.15.96:0/2435935605 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb39c004300 con 0x7fb3ac109110 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.507+0000 7fb3b133f700 1 -- 172.21.15.96:0/2435935605 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb39c011af0 con 0x7fb3ac109110 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2435935605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 msgr2=0x7fb3ac10b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2435935605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac10b500 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb39c0098a0 tx=0x7fb39c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2435935605 shutdown_connections 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2435935605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac10b500 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2435935605 >> 172.21.15.96:0/2435935605 conn(0x7fb3ac102b20 msgr2=0x7fb3ac104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2435935605 shutdown_connections 2024-04-25T20:46:43.654 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.508+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2435935605 wait complete. 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b45a5700 1 Processor -- start 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b45a5700 1 -- start start 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b45a5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac19c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b45a5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3ac10c940 con 0x7fb3ac109110 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b2341700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac19c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b2341700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac19c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56022/0 (socket says 172.21.15.96:56022) 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b2341700 1 -- 172.21.15.96:0/2900648368 learned_addr learned my addr 172.21.15.96:0/2900648368 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.509+0000 7fb3b2341700 1 -- 172.21.15.96:0/2900648368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3ac19ca60 con 0x7fb3ac109110 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.510+0000 7fb3b2341700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac19c520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009fb0 tx=0x7fb39c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.510+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb39c0036a0 con 0x7fb3ac109110 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.510+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3ac19ccf0 con 0x7fb3ac109110 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.510+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb39c01a430 con 0x7fb3ac109110 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.510+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb39c019550 con 0x7fb3ac109110 2024-04-25T20:46:43.655 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.510+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3ac19f9e0 con 0x7fb3ac109110 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.511+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fb39c019800 con 0x7fb3ac109110 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.511+0000 7fb3a37fe700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.511+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 --> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb3ac0640a0 con 0x7fb3980395b0 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.511+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb39c04da00 con 0x7fb3ac109110 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.511+0000 7fb3b1b40700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 msgr2=0x7fb39803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.511+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.712+0000 7fb3b1b40700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 msgr2=0x7fb39803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:37.712+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:38.113+0000 7fb3b1b40700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 msgr2=0x7fb39803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:38.113+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:38.915+0000 7fb3b1b40700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 msgr2=0x7fb39803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:38.915+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:40.516+0000 7fb3b1b40700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 msgr2=0x7fb39803ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/3915899740 2024-04-25T20:46:43.656 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:40.516+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:41.971+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7fb39c02d960 con 0x7fb3ac109110 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:41.971+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 msgr2=0x7fb39803ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:41.971+0000 7fb3a37fe700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.972+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7fb39c04f2c0 con 0x7fb3ac109110 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.972+0000 7fb3a37fe700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fb39803ccb0 0x7fb39803f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.972+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb39c024e50 con 0x7fb39803ccb0 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.972+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fb39803ccb0 0x7fb39803f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.973+0000 7fb3b1b40700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fb39803ccb0 0x7fb39803f0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8003a10 tx=0x7fb3a80092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.973+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7fb39c024e50 con 0x7fb39803ccb0 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fb3ac103d40 con 0x7fb39803ccb0 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3a37fe700 1 -- 172.21.15.96:0/2900648368 <== mgr.14120 v2:172.21.15.96:6800/1894355014 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fb3ac103d40 con 0x7fb39803ccb0 2024-04-25T20:46:43.657 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fb39803ccb0 msgr2=0x7fb39803f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fb39803ccb0 0x7fb39803f0a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8003a10 tx=0x7fb3a80092b0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 msgr2=0x7fb3ac19c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac19c520 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb39c009fb0 tx=0x7fb39c011940 comp rx=0 tx=0).stop 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 shutdown_connections 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fb39803ccb0 0x7fb39803f0a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3ac109110 0x7fb3ac19c520 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 --2- 172.21.15.96:0/2900648368 >> [v2:172.21.15.96:6800/3915899740,v1:172.21.15.96:6801/3915899740] conn(0x7fb3980395b0 0x7fb39803ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 >> 172.21.15.96:0/2900648368 conn(0x7fb3ac102b20 msgr2=0x7fb3ac103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 shutdown_connections 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:42.980+0000 7fb3b45a5700 1 -- 172.21.15.96:0/2900648368 wait complete. 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 5 is available 2024-04-25T20:46:43.658 INFO:teuthology.orchestra.run.smithi096.stdout:Setting orchestrator backend to cephadm... 2024-04-25T20:46:44.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: [25/Apr/2024:20:46:42] ENGINE Bus STARTING 2024-04-25T20:46:44.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T20:46:44.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T20:46:44.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: [25/Apr/2024:20:46:43] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T20:46:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: [25/Apr/2024:20:46:43] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T20:46:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: [25/Apr/2024:20:46:43] ENGINE Bus STARTED 2024-04-25T20:46:44.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:44 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:46:44.907 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.077+0000 7f3e3772e700 1 Processor -- start 2024-04-25T20:46:44.907 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.077+0000 7f3e3772e700 1 -- start start 2024-04-25T20:46:44.907 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.077+0000 7f3e3772e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e301070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:44.907 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.077+0000 7f3e3772e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e301076c0 con 0x7f3e30106cd0 2024-04-25T20:46:44.907 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e354ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e301070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e354ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e301070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56090/0 (socket says 172.21.15.96:56090) 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e354ca700 1 -- 172.21.15.96:0/3880721020 learned_addr learned my addr 172.21.15.96:0/3880721020 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e354ca700 1 -- 172.21.15.96:0/3880721020 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e30107ed0 con 0x7f3e30106cd0 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e354ca700 1 --2- 172.21.15.96:0/3880721020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e301070f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009e80 tx=0x7f3e2800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f42bce5eff510ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e2ffff700 1 -- 172.21.15.96:0/3880721020 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e28017070 con 0x7f3e30106cd0 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e2ffff700 1 -- 172.21.15.96:0/3880721020 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3e2800b8c0 con 0x7f3e30106cd0 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.078+0000 7f3e2ffff700 1 -- 172.21.15.96:0/3880721020 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e28004490 con 0x7f3e30106cd0 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 -- 172.21.15.96:0/3880721020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 msgr2=0x7f3e301070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 --2- 172.21.15.96:0/3880721020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e301070f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009e80 tx=0x7f3e2800b330 comp rx=0 tx=0).stop 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 -- 172.21.15.96:0/3880721020 shutdown_connections 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 --2- 172.21.15.96:0/3880721020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e301070f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 -- 172.21.15.96:0/3880721020 >> 172.21.15.96:0/3880721020 conn(0x7f3e301022b0 msgr2=0x7f3e301046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 -- 172.21.15.96:0/3880721020 shutdown_connections 2024-04-25T20:46:44.908 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.079+0000 7f3e3772e700 1 -- 172.21.15.96:0/3880721020 wait complete. 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e3772e700 1 Processor -- start 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e3772e700 1 -- start start 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e3772e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e3019c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e3772e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e3010b100 con 0x7f3e30106cd0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e354ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e3019c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e354ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e3019c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56096/0 (socket says 172.21.15.96:56096) 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e354ca700 1 -- 172.21.15.96:0/1286072381 learned_addr learned my addr 172.21.15.96:0/1286072381 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e354ca700 1 -- 172.21.15.96:0/1286072381 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e3019cb60 con 0x7f3e30106cd0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e354ca700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e3019c620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b7b0 tx=0x7f3e2800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e28020020 con 0x7f3e30106cd0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.080+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3e28012430 con 0x7f3e30106cd0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.081+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e3019cdf0 con 0x7f3e30106cd0 2024-04-25T20:46:44.909 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.081+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e28017040 con 0x7f3e30106cd0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.081+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e3019fae0 con 0x7f3e30106cd0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.081+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f3e28023020 con 0x7f3e30106cd0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.082+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e3010b280 con 0x7f3e30106cd0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.082+0000 7f3e2e7fc700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f3e180395b0 0x7f3e1803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.082+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e2800f050 con 0x7f3e30106cd0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.082+0000 7f3e34cc9700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f3e180395b0 0x7f3e1803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.082+0000 7f3e34cc9700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f3e180395b0 0x7f3e1803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e24006fd0 tx=0x7f3e24006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.087+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3e2802a020 con 0x7f3e30106cd0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.301+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f3e3002e3a0 con 0x7f3e180395b0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.306+0000 7f3e2e7fc700 1 -- 172.21.15.96:0/1286072381 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3e3002e3a0 con 0x7f3e180395b0 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f3e180395b0 msgr2=0x7f3e1803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:44.910 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f3e180395b0 0x7f3e1803ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3e24006fd0 tx=0x7f3e24006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 msgr2=0x7f3e3019c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e3019c620 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3e2800b7b0 tx=0x7f3e2800bd50 comp rx=0 tx=0).stop 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 shutdown_connections 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f3e180395b0 0x7f3e1803ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 --2- 172.21.15.96:0/1286072381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e30106cd0 0x7f3e3019c620 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 >> 172.21.15.96:0/1286072381 conn(0x7f3e301022b0 msgr2=0x7f3e301045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 shutdown_connections 2024-04-25T20:46:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:44.308+0000 7f3e3772e700 1 -- 172.21.15.96:0/1286072381 wait complete. 2024-04-25T20:46:45.151 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:45 smithi096 ceph-mon[29288]: mgrmap e8: x(active, since 2s) 2024-04-25T20:46:45.151 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:45 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:45.151 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:45 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:46:46.239 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:46 smithi096 ceph-mon[29288]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:46.247 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-25T20:46:46.247 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.336+0000 7f153ced4700 1 Processor -- start 2024-04-25T20:46:46.247 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.336+0000 7f153ced4700 1 -- start start 2024-04-25T20:46:46.247 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153ced4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f1538108b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153ced4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15381090d0 con 0x7f15381086e0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f1538108b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f1538108b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56104/0 (socket says 172.21.15.96:56104) 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153659c700 1 -- 172.21.15.96:0/431733053 learned_addr learned my addr 172.21.15.96:0/431733053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153659c700 1 -- 172.21.15.96:0/431733053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1538109890 con 0x7f15381086e0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153659c700 1 --2- 172.21.15.96:0/431733053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f1538108b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1530009e80 tx=0x7f153000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d5c30db49c9c26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.337+0000 7f153559a700 1 -- 172.21.15.96:0/431733053 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530017070 con 0x7f15381086e0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153559a700 1 -- 172.21.15.96:0/431733053 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f153000b8c0 con 0x7f15381086e0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153559a700 1 -- 172.21.15.96:0/431733053 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530004490 con 0x7f15381086e0 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 -- 172.21.15.96:0/431733053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 msgr2=0x7f1538108b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 --2- 172.21.15.96:0/431733053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f1538108b00 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1530009e80 tx=0x7f153000b330 comp rx=0 tx=0).stop 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 -- 172.21.15.96:0/431733053 shutdown_connections 2024-04-25T20:46:46.248 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 --2- 172.21.15.96:0/431733053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f1538108b00 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 -- 172.21.15.96:0/431733053 >> 172.21.15.96:0/431733053 conn(0x7f1538103ca0 msgr2=0x7f15381060c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 -- 172.21.15.96:0/431733053 shutdown_connections 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.338+0000 7f153ced4700 1 -- 172.21.15.96:0/431733053 wait complete. 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153ced4700 1 Processor -- start 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153ced4700 1 -- start start 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153ced4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f153819c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153ced4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f153810cac0 con 0x7f15381086e0 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f153819c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f153819c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56116/0 (socket says 172.21.15.96:56116) 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.339+0000 7f153659c700 1 -- 172.21.15.96:0/2368888596 learned_addr learned my addr 172.21.15.96:0/2368888596 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f153659c700 1 -- 172.21.15.96:0/2368888596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153819ca30 con 0x7f15381086e0 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f153659c700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f153819c4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f153000b7b0 tx=0x7f153000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530020020 con 0x7f15381086e0 2024-04-25T20:46:46.249 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1530012430 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153819ccc0 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530017040 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.340+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f153819f9b0 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.341+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f1530023020 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.341+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15380640a0 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.341+0000 7f15277fe700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f1520039560 0x7f152003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.341+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f153000f050 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.341+0000 7f1535d9b700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f1520039560 0x7f152003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.341+0000 7f1535d9b700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f1520039560 0x7f152003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f152c006fd0 tx=0x7f152c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.346+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f153002a020 con 0x7f15381086e0 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.544+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f1538104e50 con 0x7f1520039560 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.544+0000 7f15277fe700 1 -- 172.21.15.96:0/2368888596 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f1538104e50 con 0x7f1520039560 2024-04-25T20:46:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.546+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f1520039560 msgr2=0x7f152003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.546+0000 7f153ced4700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f1520039560 0x7f152003ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f152c006fd0 tx=0x7f152c006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.546+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 msgr2=0x7f153819c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.546+0000 7f153ced4700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f153819c4f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f153000b7b0 tx=0x7f153000be20 comp rx=0 tx=0).stop 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.547+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 shutdown_connections 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.547+0000 7f153ced4700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f1520039560 0x7f152003ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.547+0000 7f153ced4700 1 --2- 172.21.15.96:0/2368888596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15381086e0 0x7f153819c4f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.547+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 >> 172.21.15.96:0/2368888596 conn(0x7f1538103ca0 msgr2=0x7f1538104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.547+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 shutdown_connections 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:45.547+0000 7f153ced4700 1 -- 172.21.15.96:0/2368888596 wait complete. 2024-04-25T20:46:46.251 INFO:teuthology.orchestra.run.smithi096.stdout:Generating ssh key... 2024-04-25T20:46:47.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-mon[29288]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:47.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: Generating public/private rsa key pair. 2024-04-25T20:46:47.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: Your identification has been saved in /tmp/tmpbtvjjs7k/key. 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: Your public key has been saved in /tmp/tmpbtvjjs7k/key.pub. 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: The key fingerprint is: 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: SHA256:12x84tQeSPP0z+HpK7qsWFfgPXPsMAigGCa3A8kGLKA ceph-b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: The key's randomart image is: 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: +---[RSA 3072]----+ 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: |Bo.+ . | 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: |o== + . . | 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: |E + . . .o . | 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: | . o=+*.. | 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: | S .oOB=+.| 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: | . +.+Oo+| 2024-04-25T20:46:47.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: | . .. .+o| 2024-04-25T20:46:47.251 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: | o o .. | 2024-04-25T20:46:47.251 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: | . ..+o .o.| 2024-04-25T20:46:47.251 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:46:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: +----[SHA256]-----+ 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe5b8b7700 1 Processor -- start 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe5b8b7700 1 -- start start 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe5b8b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe54108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe5b8b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe54109060 con 0x7efe54108670 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe59653700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe54108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe59653700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe54108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45182/0 (socket says 172.21.15.96:45182) 2024-04-25T20:46:47.620 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe59653700 1 -- 172.21.15.96:0/2224843434 learned_addr learned my addr 172.21.15.96:0/2224843434 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.657+0000 7efe59653700 1 -- 172.21.15.96:0/2224843434 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe54109870 con 0x7efe54108670 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe59653700 1 --2- 172.21.15.96:0/2224843434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe54108a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe440098a0 tx=0x7efe44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9e43598e379782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe4bfff700 1 -- 172.21.15.96:0/2224843434 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe44004160 con 0x7efe54108670 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe4bfff700 1 -- 172.21.15.96:0/2224843434 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe44004300 con 0x7efe54108670 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe4bfff700 1 -- 172.21.15.96:0/2224843434 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe44011af0 con 0x7efe54108670 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 -- 172.21.15.96:0/2224843434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 msgr2=0x7efe54108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 --2- 172.21.15.96:0/2224843434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe54108a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe440098a0 tx=0x7efe44009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 -- 172.21.15.96:0/2224843434 shutdown_connections 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 --2- 172.21.15.96:0/2224843434 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe54108a90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 -- 172.21.15.96:0/2224843434 >> 172.21.15.96:0/2224843434 conn(0x7efe54103c30 msgr2=0x7efe54106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 -- 172.21.15.96:0/2224843434 shutdown_connections 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.658+0000 7efe5b8b7700 1 -- 172.21.15.96:0/2224843434 wait complete. 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe5b8b7700 1 Processor -- start 2024-04-25T20:46:47.621 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe5b8b7700 1 -- start start 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe5b8b7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe5419c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe5b8b7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe5410caa0 con 0x7efe54108670 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe59653700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe5419c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe59653700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe5419c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45186/0 (socket says 172.21.15.96:45186) 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.659+0000 7efe59653700 1 -- 172.21.15.96:0/849888479 learned_addr learned my addr 172.21.15.96:0/849888479 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe59653700 1 -- 172.21.15.96:0/849888479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5419ca90 con 0x7efe54108670 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe59653700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe5419c550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe44009fb0 tx=0x7efe44011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe440036a0 con 0x7efe54108670 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5419cd20 con 0x7efe54108670 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7efe4401a430 con 0x7efe54108670 2024-04-25T20:46:47.622 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe44019550 con 0x7efe54108670 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.660+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5419fa10 con 0x7efe54108670 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.661+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efe44019850 con 0x7efe54108670 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.661+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe540640a0 con 0x7efe54108670 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.661+0000 7efe4a7fc700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7efe40039560 0x7efe4003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.661+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe4404da50 con 0x7efe54108670 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.661+0000 7efe58e52700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7efe40039560 0x7efe4003ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.661+0000 7efe58e52700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7efe40039560 0x7efe4003ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe50006fd0 tx=0x7efe50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.666+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe4401a5e0 con 0x7efe54108670 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:46.863+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7efe54104ed0 con 0x7efe40039560 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.059+0000 7efe4a7fc700 1 -- 172.21.15.96:0/849888479 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efe54104ed0 con 0x7efe40039560 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7efe40039560 msgr2=0x7efe4003ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7efe40039560 0x7efe4003ba20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe50006fd0 tx=0x7efe50006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:47.623 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 msgr2=0x7efe5419c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe5419c550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7efe44009fb0 tx=0x7efe44011940 comp rx=0 tx=0).stop 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 shutdown_connections 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7efe40039560 0x7efe4003ba20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 --2- 172.21.15.96:0/849888479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe54108670 0x7efe5419c550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 >> 172.21.15.96:0/849888479 conn(0x7efe54103c30 msgr2=0x7efe541047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 shutdown_connections 2024-04-25T20:46:47.624 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:47.061+0000 7efe5b8b7700 1 -- 172.21.15.96:0/849888479 wait complete. 2024-04-25T20:46:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:48 smithi096 ceph-mon[29288]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:48 smithi096 ceph-mon[29288]: Generating ssh key... 2024-04-25T20:46:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:48 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:48.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:48 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f034b443700 1 Processor -- start 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f034b443700 1 -- start start 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f034b443700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f0344107e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f034b443700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0344108420 con 0x7f0344107a30 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f03491df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f0344107e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f03491df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f0344107e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45200/0 (socket says 172.21.15.96:45200) 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f03491df700 1 -- 172.21.15.96:0/1031165093 learned_addr learned my addr 172.21.15.96:0/1031165093 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.063+0000 7f03491df700 1 -- 172.21.15.96:0/1031165093 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0344108750 con 0x7f0344107a30 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f03491df700 1 --2- 172.21.15.96:0/1031165093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f0344107e50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f034000bb40 tx=0x7f034000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1310e9122a27ea69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:48.877 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f033bfff700 1 -- 172.21.15.96:0/1031165093 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f034000dbe0 con 0x7f0344107a30 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f033bfff700 1 -- 172.21.15.96:0/1031165093 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f03400044c0 con 0x7f0344107a30 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f033bfff700 1 -- 172.21.15.96:0/1031165093 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0340010470 con 0x7f0344107a30 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f034b443700 1 -- 172.21.15.96:0/1031165093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 msgr2=0x7f0344107e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f034b443700 1 --2- 172.21.15.96:0/1031165093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f0344107e50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f034000bb40 tx=0x7f034000be50 comp rx=0 tx=0).stop 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f034b443700 1 -- 172.21.15.96:0/1031165093 shutdown_connections 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f034b443700 1 --2- 172.21.15.96:0/1031165093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f0344107e50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.064+0000 7f034b443700 1 -- 172.21.15.96:0/1031165093 >> 172.21.15.96:0/1031165093 conn(0x7f03441022b0 msgr2=0x7f03441046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f034b443700 1 -- 172.21.15.96:0/1031165093 shutdown_connections 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f034b443700 1 -- 172.21.15.96:0/1031165093 wait complete. 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f034b443700 1 Processor -- start 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f034b443700 1 -- start start 2024-04-25T20:46:48.878 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f034b443700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f034419a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f034b443700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0344106590 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.065+0000 7f03491df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f034419a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f03491df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f034419a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45202/0 (socket says 172.21.15.96:45202) 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f03491df700 1 -- 172.21.15.96:0/1257996316 learned_addr learned my addr 172.21.15.96:0/1257996316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f03491df700 1 -- 172.21.15.96:0/1257996316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034419a810 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f03491df700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f034419a2d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f034000d650 tx=0x7f0340003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0340027070 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0340010c00 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034419aaa0 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0340022ba0 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.066+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034419d790 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.067+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f034002a070 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.067+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03440640a0 con 0x7f0344107a30 2024-04-25T20:46:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.067+0000 7f033a7fc700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f03300395b0 0x7f033003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.067+0000 7f03489de700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f03300395b0 0x7f033003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.067+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f034001f070 con 0x7f0344107a30 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.068+0000 7f03489de700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f03300395b0 0x7f033003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0334006fd0 tx=0x7f0334006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.072+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0340053030 con 0x7f0344107a30 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.270+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f03440754c0 con 0x7f03300395b0 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.271+0000 7f033a7fc700 1 -- 172.21.15.96:0/1257996316 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f03440754c0 con 0x7f03300395b0 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f03300395b0 msgr2=0x7f033003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f03300395b0 0x7f033003ba70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0334006fd0 tx=0x7f0334006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 msgr2=0x7f034419a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f034419a2d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f034000d650 tx=0x7f0340003ff0 comp rx=0 tx=0).stop 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 shutdown_connections 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f03300395b0 0x7f033003ba70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 --2- 172.21.15.96:0/1257996316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0344107a30 0x7f034419a2d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:48.880 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 >> 172.21.15.96:0/1257996316 conn(0x7f03441022b0 msgr2=0x7f0344190910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:48.881 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 shutdown_connections 2024-04-25T20:46:48.881 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:48.273+0000 7f034b443700 1 -- 172.21.15.96:0/1257996316 wait complete. 2024-04-25T20:46:48.881 INFO:teuthology.orchestra.run.smithi096.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-25T20:46:48.881 INFO:teuthology.orchestra.run.smithi096.stdout:Adding key to root@localhost authorized_keys... 2024-04-25T20:46:48.881 INFO:teuthology.orchestra.run.smithi096.stdout:Adding host smithi096... 2024-04-25T20:46:49.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:49 smithi096 ceph-mon[29288]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:50.368 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:50 smithi096 ceph-mon[29288]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:50.368 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:50 smithi096 ceph-mon[29288]: Deploying cephadm binary to smithi096 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Added host 'smithi096' with addr '172.21.15.96' 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.275+0000 7f7e31387700 1 Processor -- start 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.275+0000 7f7e31387700 1 -- start start 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.275+0000 7f7e31387700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.275+0000 7f7e31387700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c109050 con 0x7f7e2c108660 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e2affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e2affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45218/0 (socket says 172.21.15.96:45218) 2024-04-25T20:46:54.260 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e2affd700 1 -- 172.21.15.96:0/3432376548 learned_addr learned my addr 172.21.15.96:0/3432376548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e2affd700 1 -- 172.21.15.96:0/3432376548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c109810 con 0x7f7e2c108660 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e2affd700 1 --2- 172.21.15.96:0/3432376548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c108a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009e80 tx=0x7f7e1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728dea4d3bed20a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e29ffb700 1 -- 172.21.15.96:0/3432376548 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c017070 con 0x7f7e2c108660 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e29ffb700 1 -- 172.21.15.96:0/3432376548 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e1c00b8c0 con 0x7f7e2c108660 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.276+0000 7f7e29ffb700 1 -- 172.21.15.96:0/3432376548 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c004490 con 0x7f7e2c108660 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 -- 172.21.15.96:0/3432376548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 msgr2=0x7f7e2c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 --2- 172.21.15.96:0/3432376548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c108a80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c009e80 tx=0x7f7e1c00b330 comp rx=0 tx=0).stop 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 -- 172.21.15.96:0/3432376548 shutdown_connections 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 --2- 172.21.15.96:0/3432376548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c108a80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 -- 172.21.15.96:0/3432376548 >> 172.21.15.96:0/3432376548 conn(0x7f7e2c103c40 msgr2=0x7f7e2c106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 -- 172.21.15.96:0/3432376548 shutdown_connections 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 -- 172.21.15.96:0/3432376548 wait complete. 2024-04-25T20:46:54.261 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.277+0000 7f7e31387700 1 Processor -- start 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e31387700 1 -- start start 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e31387700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c19c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e31387700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7e2c10ca40 con 0x7f7e2c108660 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e2affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c19c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e2affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c19c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45222/0 (socket says 172.21.15.96:45222) 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e2affd700 1 -- 172.21.15.96:0/2388294099 learned_addr learned my addr 172.21.15.96:0/2388294099 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e2affd700 1 -- 172.21.15.96:0/2388294099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2c19ca00 con 0x7f7e2c108660 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e2affd700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c19c4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c004a60 tx=0x7f7e1c00bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c020020 con 0x7f7e2c108660 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7e1c012430 con 0x7f7e2c108660 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2c19cc90 con 0x7f7e2c108660 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7e1c017040 con 0x7f7e2c108660 2024-04-25T20:46:54.262 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.278+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e2c19f980 con 0x7f7e2c108660 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.279+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f7e1c023020 con 0x7f7e2c108660 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.279+0000 7f7e13fff700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f7e140395b0 0x7f7e1403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.279+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7e1c00f050 con 0x7f7e2c108660 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.280+0000 7f7e2a7fc700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f7e140395b0 0x7f7e1403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.280+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e2c10cbc0 con 0x7f7e2c108660 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.280+0000 7f7e2a7fc700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f7e140395b0 0x7f7e1403ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e20006fd0 tx=0x7f7e20006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.285+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7e1c02a020 con 0x7f7e2c108660 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:49.490+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "addr": "172.21.15.96", "target": ["mon-mgr", ""]}) -- 0x7f7e2c104e10 con 0x7f7e140395b0 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.588+0000 7f7e13fff700 1 -- 172.21.15.96:0/2388294099 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f7e2c104e10 con 0x7f7e140395b0 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f7e140395b0 msgr2=0x7f7e1403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f7e140395b0 0x7f7e1403ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7e20006fd0 tx=0x7f7e20006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 msgr2=0x7f7e2c19c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:54.263 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c19c4c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c004a60 tx=0x7f7e1c00bd50 comp rx=0 tx=0).stop 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 shutdown_connections 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f7e140395b0 0x7f7e1403ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 --2- 172.21.15.96:0/2388294099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7e2c108660 0x7f7e2c19c4c0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 >> 172.21.15.96:0/2388294099 conn(0x7f7e2c103c40 msgr2=0x7f7e2c104700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 shutdown_connections 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:53.591+0000 7f7e31387700 1 -- 172.21.15.96:0/2388294099 wait complete. 2024-04-25T20:46:54.264 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mon service... 2024-04-25T20:46:54.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:54 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:54.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:54 smithi096 ceph-mon[29288]: Added host smithi096 2024-04-25T20:46:54.972 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:54 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:46:55.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-25T20:46:55.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a87edf700 1 Processor -- start 2024-04-25T20:46:55.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a87edf700 1 -- start start 2024-04-25T20:46:55.602 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a87edf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a80106d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a87edf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a80107310 con 0x7f5a80106920 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a85c7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a80106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a85c7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a80106d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45226/0 (socket says 172.21.15.96:45226) 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.748+0000 7f5a85c7b700 1 -- 172.21.15.96:0/3896824142 learned_addr learned my addr 172.21.15.96:0/3896824142 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a85c7b700 1 -- 172.21.15.96:0/3896824142 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a801098b0 con 0x7f5a80106920 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a85c7b700 1 --2- 172.21.15.96:0/3896824142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a80106d40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0098a0 tx=0x7f5a7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9ced59ded2fd26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a84c79700 1 -- 172.21.15.96:0/3896824142 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a7c004160 con 0x7f5a80106920 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a84c79700 1 -- 172.21.15.96:0/3896824142 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5a7c004300 con 0x7f5a80106920 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a84c79700 1 -- 172.21.15.96:0/3896824142 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a7c011af0 con 0x7f5a80106920 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a87edf700 1 -- 172.21.15.96:0/3896824142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 msgr2=0x7f5a80106d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.749+0000 7f5a87edf700 1 --2- 172.21.15.96:0/3896824142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a80106d40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c0098a0 tx=0x7f5a7c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 -- 172.21.15.96:0/3896824142 shutdown_connections 2024-04-25T20:46:55.603 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 --2- 172.21.15.96:0/3896824142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a80106d40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 -- 172.21.15.96:0/3896824142 >> 172.21.15.96:0/3896824142 conn(0x7f5a801022b0 msgr2=0x7f5a801046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 -- 172.21.15.96:0/3896824142 shutdown_connections 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 -- 172.21.15.96:0/3896824142 wait complete. 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 Processor -- start 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 -- start start 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a8019c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a87edf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a8010cae0 con 0x7f5a80106920 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.750+0000 7f5a85c7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a8019c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a85c7b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a8019c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45238/0 (socket says 172.21.15.96:45238) 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a85c7b700 1 -- 172.21.15.96:0/614244847 learned_addr learned my addr 172.21.15.96:0/614244847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a85c7b700 1 -- 172.21.15.96:0/614244847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a8019cae0 con 0x7f5a80106920 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a85c7b700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a8019c5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009fb0 tx=0x7f5a7c011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:55.604 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a7c0036a0 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5a7c01a430 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a8019cd70 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5a7c019550 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.751+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a8019fa60 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.752+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5a7c019850 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.752+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a800640a0 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.752+0000 7f5a76ffd700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5a6c039560 0x7f5a6c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.752+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5a7c04d8c0 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.752+0000 7f5a8547a700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5a6c039560 0x7f5a6c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.753+0000 7f5a8547a700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5a6c039560 0x7f5a6c03ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006fd0 tx=0x7f5a70006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.757+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5a7c020240 con 0x7f5a80106920 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.970+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f5a801034d0 con 0x7f5a6c039560 2024-04-25T20:46:55.605 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.978+0000 7f5a76ffd700 1 -- 172.21.15.96:0/614244847 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5a801034d0 con 0x7f5a6c039560 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5a6c039560 msgr2=0x7f5a6c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5a6c039560 0x7f5a6c03ba20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5a70006fd0 tx=0x7f5a70006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 msgr2=0x7f5a8019c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a8019c5a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5a7c009fb0 tx=0x7f5a7c011940 comp rx=0 tx=0).stop 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 shutdown_connections 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5a6c039560 0x7f5a6c03ba20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 --2- 172.21.15.96:0/614244847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a80106920 0x7f5a8019c5a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 >> 172.21.15.96:0/614244847 conn(0x7f5a801022b0 msgr2=0x7f5a80102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 shutdown_connections 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:54.980+0000 7f5a87edf700 1 -- 172.21.15.96:0/614244847 wait complete. 2024-04-25T20:46:55.606 INFO:teuthology.orchestra.run.smithi096.stdout:Deploying unmanaged mgr service... 2024-04-25T20:46:56.216 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:55 smithi096 ceph-mon[29288]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:56.216 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:55 smithi096 ceph-mon[29288]: Saving service mon spec with placement count:5 2024-04-25T20:46:56.216 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:55 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:57.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:57 smithi096 ceph-mon[29288]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:46:57.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:57 smithi096 ceph-mon[29288]: Saving service mgr spec with placement count:2 2024-04-25T20:46:57.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:57 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:57.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-25T20:46:57.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f680e536700 1 Processor -- start 2024-04-25T20:46:57.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f680e536700 1 -- start start 2024-04-25T20:46:57.784 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f680e536700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f6808108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f680e536700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6808109060 con 0x7f6808108670 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f6807fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f6808108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f6807fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f6808108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45240/0 (socket says 172.21.15.96:45240) 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f6807fff700 1 -- 172.21.15.96:0/2959963200 learned_addr learned my addr 172.21.15.96:0/2959963200 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f6807fff700 1 -- 172.21.15.96:0/2959963200 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6808109820 con 0x7f6808108670 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.040+0000 7f6807fff700 1 --2- 172.21.15.96:0/2959963200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f6808108a90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098a0 tx=0x7f67f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29849ed2dd9e9d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f6806ffd700 1 -- 172.21.15.96:0/2959963200 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8004160 con 0x7f6808108670 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f6806ffd700 1 -- 172.21.15.96:0/2959963200 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f67f8004300 con 0x7f6808108670 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f6806ffd700 1 -- 172.21.15.96:0/2959963200 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8011af0 con 0x7f6808108670 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 -- 172.21.15.96:0/2959963200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 msgr2=0x7f6808108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 --2- 172.21.15.96:0/2959963200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f6808108a90 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f67f80098a0 tx=0x7f67f8009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 -- 172.21.15.96:0/2959963200 shutdown_connections 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 --2- 172.21.15.96:0/2959963200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f6808108a90 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:57.785 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 -- 172.21.15.96:0/2959963200 >> 172.21.15.96:0/2959963200 conn(0x7f6808103c30 msgr2=0x7f6808106050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 -- 172.21.15.96:0/2959963200 shutdown_connections 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.041+0000 7f680e536700 1 -- 172.21.15.96:0/2959963200 wait complete. 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f680e536700 1 Processor -- start 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f680e536700 1 -- start start 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f680e536700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f680819c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f680e536700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f680810ca50 con 0x7f6808108670 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f6807fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f680819c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f6807fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f680819c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45244/0 (socket says 172.21.15.96:45244) 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f6807fff700 1 -- 172.21.15.96:0/332055980 learned_addr learned my addr 172.21.15.96:0/332055980 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.042+0000 7f6807fff700 1 -- 172.21.15.96:0/332055980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f680819ca90 con 0x7f6808108670 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.043+0000 7f6807fff700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f680819c550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67f80117b0 tx=0x7f67f8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:57.786 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.043+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8003af0 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.043+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f67f8020070 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.043+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f8019630 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.043+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f680819cd20 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.043+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f680819fa10 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.044+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f67f80198a0 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.044+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f680810cbd0 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.044+0000 7f68057fa700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f67f0039600 0x7f67f003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.044+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f67f804dc60 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.044+0000 7f68077fe700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f67f0039600 0x7f67f003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.044+0000 7f68077fe700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f67f0039600 0x7f67f003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.049+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67f8023370 con 0x7f6808108670 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.280+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f6808104e70 con 0x7f67f0039600 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.287+0000 7f68057fa700 1 -- 172.21.15.96:0/332055980 <== mgr.14120 v2:172.21.15.96:6800/1894355014 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f6808104e70 con 0x7f67f0039600 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f67f0039600 msgr2=0x7f67f003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:57.787 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f67f0039600 0x7f67f003bac0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 msgr2=0x7f680819c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f680819c550 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f67f80117b0 tx=0x7f67f8011b70 comp rx=0 tx=0).stop 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 shutdown_connections 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f67f0039600 0x7f67f003bac0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 --2- 172.21.15.96:0/332055980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6808108670 0x7f680819c550 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 >> 172.21.15.96:0/332055980 conn(0x7f6808103c30 msgr2=0x7f6808104760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.289+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 shutdown_connections 2024-04-25T20:46:57.788 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:56.290+0000 7f680e536700 1 -- 172.21.15.96:0/332055980 wait complete. 2024-04-25T20:46:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:58 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:58 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:46:59.365 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe42e57d700 1 Processor -- start 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe42e57d700 1 -- start start 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe42e57d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe42810b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe42e57d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe42807ce90 con 0x7fe4281090a0 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe427fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe42810b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe427fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe42810b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41444/0 (socket says 172.21.15.96:41444) 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe427fff700 1 -- 172.21.15.96:0/3732789134 learned_addr learned my addr 172.21.15.96:0/3732789134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe427fff700 1 -- 172.21.15.96:0/3732789134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe42810b9d0 con 0x7fe4281090a0 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.257+0000 7fe427fff700 1 --2- 172.21.15.96:0/3732789134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe42810b490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe418009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22523e9c01d09399 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.258+0000 7fe426ffd700 1 -- 172.21.15.96:0/3732789134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe418004160 con 0x7fe4281090a0 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.258+0000 7fe426ffd700 1 -- 172.21.15.96:0/3732789134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe418004300 con 0x7fe4281090a0 2024-04-25T20:46:59.366 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.258+0000 7fe426ffd700 1 -- 172.21.15.96:0/3732789134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe418011a20 con 0x7fe4281090a0 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.258+0000 7fe42e57d700 1 -- 172.21.15.96:0/3732789134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 msgr2=0x7fe42810b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.258+0000 7fe42e57d700 1 --2- 172.21.15.96:0/3732789134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe42810b490 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe4180098a0 tx=0x7fe418009bb0 comp rx=0 tx=0).stop 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 -- 172.21.15.96:0/3732789134 shutdown_connections 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 --2- 172.21.15.96:0/3732789134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe42810b490 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 -- 172.21.15.96:0/3732789134 >> 172.21.15.96:0/3732789134 conn(0x7fe428102b10 msgr2=0x7fe428104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 -- 172.21.15.96:0/3732789134 shutdown_connections 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 -- 172.21.15.96:0/3732789134 wait complete. 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 Processor -- start 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.259+0000 7fe42e57d700 1 -- start start 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe42e57d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe428108920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe42e57d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe42810c970 con 0x7fe4281090a0 2024-04-25T20:46:59.367 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe427fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe428108920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe427fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe428108920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41460/0 (socket says 172.21.15.96:41460) 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe427fff700 1 -- 172.21.15.96:0/509697689 learned_addr learned my addr 172.21.15.96:0/509697689 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe427fff700 1 -- 172.21.15.96:0/509697689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe428108e60 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe427fff700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe428108920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe418004660 tx=0x7fe418011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe418003aa0 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe41801b8f0 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe428107050 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe418023900 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.260+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe428107590 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.261+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe418023b70 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.261+0000 7fe4257fa700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fe410039600 0x7fe41003bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.262+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe41804eda0 con 0x7fe4281090a0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.262+0000 7fe4277fe700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fe410039600 0x7fe41003bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:46:59.368 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.262+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4280640a0 con 0x7fe4281090a0 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.262+0000 7fe4277fe700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fe410039600 0x7fe41003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006fd0 tx=0x7fe41c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.267+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe41801f030 con 0x7fe4281090a0 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.503+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fe428107cb0 con 0x7fe4281090a0 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.510+0000 7fe4257fa700 1 -- 172.21.15.96:0/509697689 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fe41801bde0 con 0x7fe4281090a0 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fe410039600 msgr2=0x7fe41003bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fe410039600 0x7fe41003bac0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe41c006fd0 tx=0x7fe41c006e40 comp rx=0 tx=0).stop 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 msgr2=0x7fe428108920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe428108920 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe418004660 tx=0x7fe418011aa0 comp rx=0 tx=0).stop 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 shutdown_connections 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fe410039600 0x7fe41003bac0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 --2- 172.21.15.96:0/509697689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4281090a0 0x7fe428108920 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 >> 172.21.15.96:0/509697689 conn(0x7fe428102b10 msgr2=0x7fe428103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:46:59.369 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 shutdown_connections 2024-04-25T20:46:59.370 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:46:58.513+0000 7fe42e57d700 1 -- 172.21.15.96:0/509697689 wait complete. 2024-04-25T20:46:59.625 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:46:59 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/509697689' entity='client.admin' 2024-04-25T20:47:01.177 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:01 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:47:01.177 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:01 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1170312683' entity='client.admin' 2024-04-25T20:47:01.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb4752f700 1 Processor -- start 2024-04-25T20:47:01.186 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb4752f700 1 -- start start 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb4752f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb40107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb4752f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40107f30 con 0x7fbb40107540 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb452cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb40107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb452cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb40107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41464/0 (socket says 172.21.15.96:41464) 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.227+0000 7fbb452cb700 1 -- 172.21.15.96:0/3822968177 learned_addr learned my addr 172.21.15.96:0/3822968177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.228+0000 7fbb452cb700 1 -- 172.21.15.96:0/3822968177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb40108740 con 0x7fbb40107540 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.228+0000 7fbb452cb700 1 --2- 172.21.15.96:0/3822968177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb40107960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0098a0 tx=0x7fbb3c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa2b740d6d1958a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.228+0000 7fbb37fff700 1 -- 172.21.15.96:0/3822968177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c004160 con 0x7fbb40107540 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.228+0000 7fbb37fff700 1 -- 172.21.15.96:0/3822968177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c004300 con 0x7fbb40107540 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.228+0000 7fbb37fff700 1 -- 172.21.15.96:0/3822968177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c011a20 con 0x7fbb40107540 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 -- 172.21.15.96:0/3822968177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 msgr2=0x7fbb40107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 --2- 172.21.15.96:0/3822968177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb40107960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0098a0 tx=0x7fbb3c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:01.187 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 -- 172.21.15.96:0/3822968177 shutdown_connections 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 --2- 172.21.15.96:0/3822968177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb40107960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 -- 172.21.15.96:0/3822968177 >> 172.21.15.96:0/3822968177 conn(0x7fbb40102b80 msgr2=0x7fbb40104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 -- 172.21.15.96:0/3822968177 shutdown_connections 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 -- 172.21.15.96:0/3822968177 wait complete. 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 Processor -- start 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.229+0000 7fbb4752f700 1 -- start start 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb4752f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb4019c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb4752f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb4010baf0 con 0x7fbb40107540 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb452cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb4019c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb452cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb4019c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41472/0 (socket says 172.21.15.96:41472) 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb452cb700 1 -- 172.21.15.96:0/1170312683 learned_addr learned my addr 172.21.15.96:0/1170312683 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb452cb700 1 -- 172.21.15.96:0/1170312683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019cad0 con 0x7fbb40107540 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb452cb700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb4019c590 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0119f0 tx=0x7fbb3c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:01.188 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c0039e0 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb3c01b8f0 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019cd60 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb3c0237a0 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.230+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb4019fa50 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.231+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fbb3c02c3f0 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.231+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4010bc70 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.231+0000 7fbb367fc700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fbb2c039560 0x7fbb2c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.232+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fbb3c04deb0 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.232+0000 7fbb44aca700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fbb2c039560 0x7fbb2c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.232+0000 7fbb44aca700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fbb2c039560 0x7fbb2c03ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbb30006fd0 tx=0x7fbb30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.237+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb3c01f030 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.438+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fbb4002e230 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.441+0000 7fbb367fc700 1 -- 172.21.15.96:0/1170312683 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fbb3c0202b0 con 0x7fbb40107540 2024-04-25T20:47:01.189 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fbb2c039560 msgr2=0x7fbb2c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fbb2c039560 0x7fbb2c03ba20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fbb30006fd0 tx=0x7fbb30006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 msgr2=0x7fbb4019c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb4019c590 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c0119f0 tx=0x7fbb3c011ad0 comp rx=0 tx=0).stop 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 shutdown_connections 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7fbb2c039560 0x7fbb2c03ba20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 --2- 172.21.15.96:0/1170312683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb40107540 0x7fbb4019c590 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 >> 172.21.15.96:0/1170312683 conn(0x7fbb40102b80 msgr2=0x7fbb40103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 shutdown_connections 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:00.444+0000 7fbb4752f700 1 -- 172.21.15.96:0/1170312683 wait complete. 2024-04-25T20:47:01.190 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling the dashboard module... 2024-04-25T20:47:02.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-mon[29288]: from='mgr.14120 172.21.15.96:0/3081518022' entity='mgr.x' 2024-04-25T20:47:02.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2428263641' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-25T20:47:02.424 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-mon[29288]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:02.424 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: ignoring --setuser ceph since I am not root 2024-04-25T20:47:02.425 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: ignoring --setgroup ceph since I am not root 2024-04-25T20:47:02.425 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:02.423+0000 7f5a8ab3c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:47:02.685 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:02.606+0000 7f5a8ab3c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:47:02.961 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:02.710+0000 7f5a8ab3c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:47:03.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:03.018+0000 7f5a8ab3c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:47:03.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:03.105+0000 7f5a8ab3c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.638+0000 7f0fdb5cc700 1 Processor -- start 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.638+0000 7f0fdb5cc700 1 -- start start 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.638+0000 7f0fdb5cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd410b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.638+0000 7f0fdb5cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd4063d20 con 0x7f0fd41090f0 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fd9368700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd410b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fd9368700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd410b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41484/0 (socket says 172.21.15.96:41484) 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fd9368700 1 -- 172.21.15.96:0/2439053802 learned_addr learned my addr 172.21.15.96:0/2439053802 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:03.398 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fd9368700 1 -- 172.21.15.96:0/2439053802 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd410ba20 con 0x7f0fd41090f0 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fd9368700 1 --2- 172.21.15.96:0/2439053802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd410b4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40098a0 tx=0x7f0fc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f26824d6dcf875ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fcbfff700 1 -- 172.21.15.96:0/2439053802 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4004160 con 0x7f0fd41090f0 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fcbfff700 1 -- 172.21.15.96:0/2439053802 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc4004300 con 0x7f0fd41090f0 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.639+0000 7f0fcbfff700 1 -- 172.21.15.96:0/2439053802 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4011a20 con 0x7f0fd41090f0 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2439053802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 msgr2=0x7f0fd410b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 --2- 172.21.15.96:0/2439053802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd410b4e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40098a0 tx=0x7f0fc4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2439053802 shutdown_connections 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 --2- 172.21.15.96:0/2439053802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd410b4e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2439053802 >> 172.21.15.96:0/2439053802 conn(0x7f0fd4102b80 msgr2=0x7f0fd4104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2439053802 shutdown_connections 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.640+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2439053802 wait complete. 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fdb5cc700 1 Processor -- start 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fdb5cc700 1 -- start start 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fdb5cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:03.399 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fdb5cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd410ca20 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fd9368700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fd9368700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41500/0 (socket says 172.21.15.96:41500) 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fd9368700 1 -- 172.21.15.96:0/2428263641 learned_addr learned my addr 172.21.15.96:0/2428263641 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fd9368700 1 -- 172.21.15.96:0/2428263641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd419ca10 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.641+0000 7f0fd9368700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd419c4d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4004660 tx=0x7f0fc4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.642+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4003aa0 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.642+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd419cca0 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.642+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fc401b8f0 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.642+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4023900 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.642+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd419f990 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.643+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0fc401baa0 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.643+0000 7f0fca7fc700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f0fc00395b0 0x7f0fc003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.643+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd40640a0 con 0x7f0fd41090f0 2024-04-25T20:47:03.400 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.643+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0fc404ded0 con 0x7f0fd41090f0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.643+0000 7f0fd8b67700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f0fc00395b0 0x7f0fc003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.643+0000 7f0fd8b67700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f0fc00395b0 0x7f0fc003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0006fd0 tx=0x7f0fd0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.648+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fc401f030 con 0x7f0fd41090f0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:01.915+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f0fd419ff40 con 0x7f0fd41090f0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.246+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f0fc401a3d0 con 0x7f0fd41090f0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.248+0000 7f0fca7fc700 1 -- 172.21.15.96:0/2428263641 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f0fc4023bc0 con 0x7f0fd41090f0 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f0fc00395b0 msgr2=0x7f0fc003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f0fc00395b0 0x7f0fc003ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0006fd0 tx=0x7f0fd0006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 msgr2=0x7f0fd419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd419c4d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4004660 tx=0x7f0fc4011aa0 comp rx=0 tx=0).stop 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 shutdown_connections 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f0fc00395b0 0x7f0fc003ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 --2- 172.21.15.96:0/2428263641 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fd41090f0 0x7f0fd419c4d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:03.401 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 >> 172.21.15.96:0/2428263641 conn(0x7f0fd4102b80 msgr2=0x7f0fd4103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:03.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 shutdown_connections 2024-04-25T20:47:03.402 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:02.249+0000 7f0fdb5cc700 1 -- 172.21.15.96:0/2428263641 wait complete. 2024-04-25T20:47:03.516 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:03 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2428263641' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-25T20:47:03.516 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:03 smithi096 ceph-mon[29288]: mgrmap e9: x(active, since 20s) 2024-04-25T20:47:04.493 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:04 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4014208560' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-25T20:47:04.494 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:04.318+0000 7f5a8ab3c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:47:04.494 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:04.492+0000 7f5a8ab3c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5186eff700 1 Processor -- start 2024-04-25T20:47:04.730 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5186eff700 1 -- start start 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5186eff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180107960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5186eff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5180107f30 con 0x7f5180107540 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5184c9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180107960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5184c9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180107960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41522/0 (socket says 172.21.15.96:41522) 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5184c9b700 1 -- 172.21.15.96:0/1398473570 learned_addr learned my addr 172.21.15.96:0/1398473570 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.867+0000 7f5184c9b700 1 -- 172.21.15.96:0/1398473570 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5180108740 con 0x7f5180107540 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f5184c9b700 1 --2- 172.21.15.96:0/1398473570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180107960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51740098a0 tx=0x7f5174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dfc311418265a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f517f7fe700 1 -- 172.21.15.96:0/1398473570 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5174004160 con 0x7f5180107540 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f517f7fe700 1 -- 172.21.15.96:0/1398473570 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5174004300 con 0x7f5180107540 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f517f7fe700 1 -- 172.21.15.96:0/1398473570 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5174011a20 con 0x7f5180107540 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f5186eff700 1 -- 172.21.15.96:0/1398473570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 msgr2=0x7f5180107960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f5186eff700 1 --2- 172.21.15.96:0/1398473570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180107960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f51740098a0 tx=0x7f5174009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:04.731 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f5186eff700 1 -- 172.21.15.96:0/1398473570 shutdown_connections 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f5186eff700 1 --2- 172.21.15.96:0/1398473570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180107960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.868+0000 7f5186eff700 1 -- 172.21.15.96:0/1398473570 >> 172.21.15.96:0/1398473570 conn(0x7f5180102b80 msgr2=0x7f5180104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5186eff700 1 -- 172.21.15.96:0/1398473570 shutdown_connections 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5186eff700 1 -- 172.21.15.96:0/1398473570 wait complete. 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5186eff700 1 Processor -- start 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5186eff700 1 -- start start 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5186eff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180063960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5186eff700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f518010baf0 con 0x7f5180107540 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5184c9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180063960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5184c9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180063960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41524/0 (socket says 172.21.15.96:41524) 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.869+0000 7f5184c9b700 1 -- 172.21.15.96:0/4014208560 learned_addr learned my addr 172.21.15.96:0/4014208560 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f5184c9b700 1 -- 172.21.15.96:0/4014208560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f518019f230 con 0x7f5180107540 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f5184c9b700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180063960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:04.732 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5174003aa0 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f517401b8f0 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f518019f400 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5174023890 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.870+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f518019f820 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.871+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f517402c3f0 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.871+0000 7f517dffb700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 0x7f517003bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.871+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f517404df30 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.871+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f518010bc70 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.871+0000 7f517ffff700 1 -- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 msgr2=0x7f517003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.871+0000 7f517ffff700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 0x7f517003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:03.876+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5174048080 con 0x7f5180107540 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.072+0000 7f517ffff700 1 -- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 msgr2=0x7f517003bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:04.733 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.072+0000 7f517ffff700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 0x7f517003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.144+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f518019ff80 con 0x7f5180107540 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.144+0000 7f517dffb700 1 -- 172.21.15.96:0/4014208560 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f517401f030 con 0x7f5180107540 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 msgr2=0x7f517003bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 0x7f517003bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 msgr2=0x7f5180063960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180063960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5174000c00 tx=0x7f5174011aa0 comp rx=0 tx=0).stop 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 shutdown_connections 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7f5170039660 0x7f517003bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 --2- 172.21.15.96:0/4014208560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5180107540 0x7f5180063960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 >> 172.21.15.96:0/4014208560 conn(0x7f5180102b80 msgr2=0x7f5180103690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 shutdown_connections 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:04.147+0000 7f5186eff700 1 -- 172.21.15.96:0/4014208560 wait complete. 2024-04-25T20:47:04.734 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for the mgr to restart... 2024-04-25T20:47:04.735 INFO:teuthology.orchestra.run.smithi096.stdout:Waiting for mgr epoch 9... 2024-04-25T20:47:04.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:04.666+0000 7f5a8ab3c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:47:05.166 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:04.965+0000 7f5a8ab3c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:47:05.166 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:05.062+0000 7f5a8ab3c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:47:05.465 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:05.271+0000 7f5a8ab3c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:47:05.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:05.464+0000 7f5a8ab3c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:47:06.486 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:06.063+0000 7f5a8ab3c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:47:06.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:06.484+0000 7f5a8ab3c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:47:06.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:06.581+0000 7f5a8ab3c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:47:07.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:06.785+0000 7f5a8ab3c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:47:07.648 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:07.307+0000 7f5a8ab3c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:47:07.648 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:07.400+0000 7f5a8ab3c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:47:07.993 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:07.647+0000 7f5a8ab3c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:47:07.993 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:07.780+0000 7f5a8ab3c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:47:07.994 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:07.875+0000 7f5a8ab3c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:47:07.994 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:07.992+0000 7f5a8ab3c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:47:08.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:08 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:08.291+0000 7f5a8ab3c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:47:09.139 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:08 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:08.917+0000 7f5a8ab3c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:47:09.138+0000 7f5a8ab3c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: Active manager daemon x restarted 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: Activating manager daemon x 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: osdmap e3: 0 total, 0 up, 0 in 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: mgrmap e10: x(active, starting, since 0.00351512s) 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:47:09.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:47:09.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:47:09.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: Manager daemon x is now available 2024-04-25T20:47:09.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:09.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:47:10.364 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:47:10.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:10 smithi096 ceph-mon[29288]: mgrmap e11: x(active, since 1.00568s) 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout { 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout } 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa35014700 1 Processor -- start 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa35014700 1 -- start start 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa35014700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa30108ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa35014700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa30109090 con 0x7faa301086a0 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa2ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa30108ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa2ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa30108ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41526/0 (socket says 172.21.15.96:41526) 2024-04-25T20:47:10.998 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.185+0000 7faa2ed9d700 1 -- 172.21.15.96:0/2256588781 learned_addr learned my addr 172.21.15.96:0/2256588781 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.186+0000 7faa2ed9d700 1 -- 172.21.15.96:0/2256588781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa301098a0 con 0x7faa301086a0 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.186+0000 7faa2ed9d700 1 --2- 172.21.15.96:0/2256588781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa30108ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa200098a0 tx=0x7faa20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65ce528c022ccb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.186+0000 7faa2dd9b700 1 -- 172.21.15.96:0/2256588781 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa20004160 con 0x7faa301086a0 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.186+0000 7faa2dd9b700 1 -- 172.21.15.96:0/2256588781 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa20004300 con 0x7faa301086a0 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.186+0000 7faa2dd9b700 1 -- 172.21.15.96:0/2256588781 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa20011a20 con 0x7faa301086a0 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 -- 172.21.15.96:0/2256588781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 msgr2=0x7faa30108ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 --2- 172.21.15.96:0/2256588781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa30108ac0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faa200098a0 tx=0x7faa20009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 -- 172.21.15.96:0/2256588781 shutdown_connections 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 --2- 172.21.15.96:0/2256588781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa30108ac0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 -- 172.21.15.96:0/2256588781 >> 172.21.15.96:0/2256588781 conn(0x7faa30103cc0 msgr2=0x7faa301060e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 -- 172.21.15.96:0/2256588781 shutdown_connections 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.187+0000 7faa35014700 1 -- 172.21.15.96:0/2256588781 wait complete. 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.188+0000 7faa35014700 1 Processor -- start 2024-04-25T20:47:10.999 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.188+0000 7faa35014700 1 -- start start 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.188+0000 7faa35014700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa3019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.188+0000 7faa35014700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faa3010cc50 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.188+0000 7faa2ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa3019c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.188+0000 7faa2ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa3019c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41528/0 (socket says 172.21.15.96:41528) 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa2ed9d700 1 -- 172.21.15.96:0/2900665799 learned_addr learned my addr 172.21.15.96:0/2900665799 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa2ed9d700 1 -- 172.21.15.96:0/2900665799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3019ca30 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa2ed9d700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa3019c4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa20000c00 tx=0x7faa20011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa20003aa0 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3019ccc0 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2001b8f0 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa20023890 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.189+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa3019f9b0 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.190+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7faa2002c3f0 con 0x7faa301086a0 2024-04-25T20:47:11.000 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.190+0000 7faa17fff700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.190+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 --> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faa1803c1d0 con 0x7faa18039600 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.190+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa2004e070 con 0x7faa301086a0 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.190+0000 7faa2e59c700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 msgr2=0x7faa1803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.190+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.391+0000 7faa2e59c700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 msgr2=0x7faa1803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.391+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.792+0000 7faa2e59c700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 msgr2=0x7faa1803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:05.792+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:06.593+0000 7faa2e59c700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 msgr2=0x7faa1803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:06.593+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:08.194+0000 7faa2e59c700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 msgr2=0x7faa1803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/1894355014 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:08.194+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:09.142+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7faa200193d0 con 0x7faa301086a0 2024-04-25T20:47:11.001 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:09.142+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 msgr2=0x7faa1803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:09.142+0000 7faa17fff700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.144+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7faa2002b3a0 con 0x7faa301086a0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.144+0000 7faa17fff700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faa1803cfe0 0x7faa1803f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.144+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7faa2001ae50 con 0x7faa1803cfe0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.144+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faa1803cfe0 0x7faa1803f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.145+0000 7faa2e59c700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faa1803cfe0 0x7faa1803f3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faa24003a10 tx=0x7faa240092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.145+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7faa2001ae50 con 0x7faa1803cfe0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.157+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7faa300640a0 con 0x7faa1803cfe0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.157+0000 7faa17fff700 1 -- 172.21.15.96:0/2900665799 <== mgr.14152 v2:172.21.15.96:6800/23858312 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7faa300640a0 con 0x7faa1803cfe0 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faa1803cfe0 msgr2=0x7faa1803f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faa1803cfe0 0x7faa1803f3d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7faa24003a10 tx=0x7faa240092b0 comp rx=0 tx=0).stop 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 msgr2=0x7faa3019c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa3019c4f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa20000c00 tx=0x7faa20011aa0 comp rx=0 tx=0).stop 2024-04-25T20:47:11.002 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 shutdown_connections 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faa1803cfe0 0x7faa1803f3d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faa301086a0 0x7faa3019c4f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 --2- 172.21.15.96:0/2900665799 >> [v2:172.21.15.96:6800/1894355014,v1:172.21.15.96:6801/1894355014] conn(0x7faa18039600 0x7faa1803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 >> 172.21.15.96:0/2900665799 conn(0x7faa30103cc0 msgr2=0x7faa301047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 shutdown_connections 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:10.158+0000 7faa35014700 1 -- 172.21.15.96:0/2900665799 wait complete. 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:mgr epoch 9 is available 2024-04-25T20:47:11.003 INFO:teuthology.orchestra.run.smithi096.stdout:Generating a dashboard self-signed certificate... 2024-04-25T20:47:11.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:11 smithi096 ceph-mon[29288]: [25/Apr/2024:20:47:09] ENGINE Bus STARTING 2024-04-25T20:47:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:11 smithi096 ceph-mon[29288]: [25/Apr/2024:20:47:09] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T20:47:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:11 smithi096 ceph-mon[29288]: [25/Apr/2024:20:47:09] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T20:47:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:11 smithi096 ceph-mon[29288]: [25/Apr/2024:20:47:09] ENGINE Bus STARTED 2024-04-25T20:47:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:11 smithi096 ceph-mon[29288]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-25T20:47:11.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:11 smithi096 ceph-mon[29288]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-25T20:47:12.469 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:12 smithi096 ceph-mon[29288]: mgrmap e12: x(active, since 2s) 2024-04-25T20:47:12.470 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:12.470 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:12.470 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:12.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-25T20:47:12.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c90c32700 1 Processor -- start 2024-04-25T20:47:12.479 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c90c32700 1 -- start start 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c90c32700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c108a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c90c32700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c109050 con 0x7f6c8c108660 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c8a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c108a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c8a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c108a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32876/0 (socket says 172.21.15.96:32876) 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c8a59c700 1 -- 172.21.15.96:0/3832940117 learned_addr learned my addr 172.21.15.96:0/3832940117 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.572+0000 7f6c8a59c700 1 -- 172.21.15.96:0/3832940117 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c109810 con 0x7f6c8c108660 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.573+0000 7f6c8a59c700 1 --2- 172.21.15.96:0/3832940117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c108a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098a0 tx=0x7f6c7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f461c0d4aa05c51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:12.480 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.573+0000 7f6c8959a700 1 -- 172.21.15.96:0/3832940117 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c004160 con 0x7f6c8c108660 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.573+0000 7f6c8959a700 1 -- 172.21.15.96:0/3832940117 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c7c004300 con 0x7f6c8c108660 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.573+0000 7f6c8959a700 1 -- 172.21.15.96:0/3832940117 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c011a20 con 0x7f6c8c108660 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.573+0000 7f6c90c32700 1 -- 172.21.15.96:0/3832940117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 msgr2=0x7f6c8c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.573+0000 7f6c90c32700 1 --2- 172.21.15.96:0/3832940117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c108a80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0098a0 tx=0x7f6c7c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 -- 172.21.15.96:0/3832940117 shutdown_connections 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 --2- 172.21.15.96:0/3832940117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c108a80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 -- 172.21.15.96:0/3832940117 >> 172.21.15.96:0/3832940117 conn(0x7f6c8c103c40 msgr2=0x7f6c8c106060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 -- 172.21.15.96:0/3832940117 shutdown_connections 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 -- 172.21.15.96:0/3832940117 wait complete. 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 Processor -- start 2024-04-25T20:47:12.481 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.574+0000 7f6c90c32700 1 -- start start 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c90c32700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c90c32700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c8c10cbc0 con 0x7f6c8c108660 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c8a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c8a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32880/0 (socket says 172.21.15.96:32880) 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c8a59c700 1 -- 172.21.15.96:0/2180148552 learned_addr learned my addr 172.21.15.96:0/2180148552 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c8a59c700 1 -- 172.21.15.96:0/2180148552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8c19ec10 con 0x7f6c8c108660 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c8a59c700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c19e6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0119f0 tx=0x7f6c7c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:12.482 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c0039e0 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c7c01b8f0 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8c19eea0 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c7c0237a0 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.575+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c8c1a1b90 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.576+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f6c7c023aa0 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.576+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c8c10cd40 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.577+0000 7f6c7b7fe700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c740395b0 0x7f6c7403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.577+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6c7c01bd60 con 0x7f6c8c108660 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.577+0000 7f6c89d9b700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c740395b0 0x7f6c7403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:12.483 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.577+0000 7f6c89d9b700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c740395b0 0x7f6c7403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c80006fd0 tx=0x7f6c80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.582+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c7c01f030 con 0x7f6c8c108660 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.794+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6c8c104d90 con 0x7f6c740395b0 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.890+0000 7f6c7b7fe700 1 -- 172.21.15.96:0/2180148552 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6c8c104d90 con 0x7f6c740395b0 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c740395b0 msgr2=0x7f6c7403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c740395b0 0x7f6c7403ba70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6c80006fd0 tx=0x7f6c80006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 msgr2=0x7f6c8c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c19e6d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c0119f0 tx=0x7f6c7c011ad0 comp rx=0 tx=0).stop 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 shutdown_connections 2024-04-25T20:47:12.484 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c740395b0 0x7f6c7403ba70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:12.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 --2- 172.21.15.96:0/2180148552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c8c108660 0x7f6c8c19e6d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:12.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.892+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 >> 172.21.15.96:0/2180148552 conn(0x7f6c8c103c40 msgr2=0x7f6c8c104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:12.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.893+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 shutdown_connections 2024-04-25T20:47:12.485 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:11.893+0000 7f6c90c32700 1 -- 172.21.15.96:0/2180148552 wait complete. 2024-04-25T20:47:12.485 INFO:teuthology.orchestra.run.smithi096.stdout:Creating initial admin user... 2024-04-25T20:47:13.415 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:13 smithi096 ceph-mon[29288]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:13.415 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$74Bma3xAxmIKpZUXde0mR.fvhsLAw.CDSma5292jgDwzk6UkPE/fC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714078033, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.928+0000 7f357f681700 1 Processor -- start 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.928+0000 7f357f681700 1 -- start start 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.928+0000 7f357f681700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578106d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.928+0000 7f357f681700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3578107330 con 0x7f3578106940 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.928+0000 7f357d41d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f357d41d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578106d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32888/0 (socket says 172.21.15.96:32888) 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f357d41d700 1 -- 172.21.15.96:0/3020316767 learned_addr learned my addr 172.21.15.96:0/3020316767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f357d41d700 1 -- 172.21.15.96:0/3020316767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35781098d0 con 0x7f3578106940 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f357d41d700 1 --2- 172.21.15.96:0/3020316767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578106d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35740098a0 tx=0x7f3574009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e389be9542640b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:14.128 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f356ffff700 1 -- 172.21.15.96:0/3020316767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3574004160 con 0x7f3578106940 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f356ffff700 1 -- 172.21.15.96:0/3020316767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3574004300 con 0x7f3578106940 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.929+0000 7f356ffff700 1 -- 172.21.15.96:0/3020316767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3574011a20 con 0x7f3578106940 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 -- 172.21.15.96:0/3020316767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 msgr2=0x7f3578106d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 --2- 172.21.15.96:0/3020316767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578106d60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f35740098a0 tx=0x7f3574009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 -- 172.21.15.96:0/3020316767 shutdown_connections 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 --2- 172.21.15.96:0/3020316767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578106d60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 -- 172.21.15.96:0/3020316767 >> 172.21.15.96:0/3020316767 conn(0x7f35781022b0 msgr2=0x7f35781046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 -- 172.21.15.96:0/3020316767 shutdown_connections 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.930+0000 7f357f681700 1 -- 172.21.15.96:0/3020316767 wait complete. 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357f681700 1 Processor -- start 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357f681700 1 -- start start 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357f681700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357f681700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f357810cc80 con 0x7f3578106940 2024-04-25T20:47:14.129 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357d41d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357d41d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578109150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32904/0 (socket says 172.21.15.96:32904) 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357d41d700 1 -- 172.21.15.96:0/3421543039 learned_addr learned my addr 172.21.15.96:0/3421543039 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357d41d700 1 -- 172.21.15.96:0/3421543039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3578109690 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.931+0000 7f357d41d700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578109150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35740119f0 tx=0x7f3574011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.932+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35740039e0 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.932+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f357401b8f0 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.932+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3578107880 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.932+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35740237a0 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.932+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3578107dc0 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.933+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f357402c3f0 con 0x7f3578106940 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.933+0000 7f356e7fc700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f3564039610 0x7f356403bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:14.130 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.933+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f357404e110 con 0x7f3578106940 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.933+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35780640a0 con 0x7f3578106940 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.933+0000 7f357cc1c700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f3564039610 0x7f356403bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.933+0000 7f357cc1c700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f3564039610 0x7f356403bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3568006fd0 tx=0x7f3568006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:12.938+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f357401f030 con 0x7f3578106940 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.158+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f3578108040 con 0x7f3564039610 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.386+0000 7f356e7fc700 1 -- 172.21.15.96:0/3421543039 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f3578108040 con 0x7f3564039610 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.388+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f3564039610 msgr2=0x7f356403bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f3564039610 0x7f356403bad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3568006fd0 tx=0x7f3568006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 msgr2=0x7f3578109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578109150 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f35740119f0 tx=0x7f3574011ad0 comp rx=0 tx=0).stop 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 shutdown_connections 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f3564039610 0x7f356403bad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 --2- 172.21.15.96:0/3421543039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3578106940 0x7f3578109150 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:14.131 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 >> 172.21.15.96:0/3421543039 conn(0x7f35781022b0 msgr2=0x7f3578102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:14.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 shutdown_connections 2024-04-25T20:47:14.132 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:13.389+0000 7f357f681700 1 -- 172.21.15.96:0/3421543039 wait complete. 2024-04-25T20:47:14.132 INFO:teuthology.orchestra.run.smithi096.stdout:Fetching dashboard port number... 2024-04-25T20:47:14.383 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:14 smithi096 ceph-mon[29288]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:14.383 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:16.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stdout 8443 2024-04-25T20:47:16.004 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbfb43700 1 Processor -- start 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbfb43700 1 -- start start 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbfb43700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb8108a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbfb43700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb8108fd0 con 0x7f4fb81085e0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbd8df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb8108a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbd8df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb8108a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32906/0 (socket says 172.21.15.96:32906) 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.000+0000 7f4fbd8df700 1 -- 172.21.15.96:0/1824418715 learned_addr learned my addr 172.21.15.96:0/1824418715 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbd8df700 1 -- 172.21.15.96:0/1824418715 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb81097e0 con 0x7f4fb81085e0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbd8df700 1 --2- 172.21.15.96:0/1824418715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb8108a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098a0 tx=0x7f4fb4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632074fcc340b469 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbc8dd700 1 -- 172.21.15.96:0/1824418715 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fb4004160 con 0x7f4fb81085e0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbc8dd700 1 -- 172.21.15.96:0/1824418715 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb4004300 con 0x7f4fb81085e0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbc8dd700 1 -- 172.21.15.96:0/1824418715 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fb4011a20 con 0x7f4fb81085e0 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1824418715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 msgr2=0x7f4fb8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:16.005 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbfb43700 1 --2- 172.21.15.96:0/1824418715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb8108a00 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098a0 tx=0x7f4fb4009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1824418715 shutdown_connections 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbfb43700 1 --2- 172.21.15.96:0/1824418715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb8108a00 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.001+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1824418715 >> 172.21.15.96:0/1824418715 conn(0x7f4fb8103ba0 msgr2=0x7f4fb8105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1824418715 shutdown_connections 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1824418715 wait complete. 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbfb43700 1 Processor -- start 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbfb43700 1 -- start start 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbfb43700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb819c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbfb43700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb810cb90 con 0x7f4fb81085e0 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbd8df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb819c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbd8df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb819c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:32914/0 (socket says 172.21.15.96:32914) 2024-04-25T20:47:16.006 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.002+0000 7f4fbd8df700 1 -- 172.21.15.96:0/1405667539 learned_addr learned my addr 172.21.15.96:0/1405667539 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4fbd8df700 1 -- 172.21.15.96:0/1405667539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb819ca20 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4fbd8df700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb819c4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40119f0 tx=0x7f4fb4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fb40039e0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb819ccb0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fb401b8f0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fb40237a0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.003+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb819f9a0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.004+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4fb402c3f0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.004+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb810cd10 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.004+0000 7f4faeffd700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4fa40395b0 0x7f4fa403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.004+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4fb404dff0 con 0x7f4fb81085e0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.004+0000 7f4fbd0de700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4fa40395b0 0x7f4fa403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:16.007 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.004+0000 7f4fbd0de700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4fa40395b0 0x7f4fa403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8006fd0 tx=0x7f4fa8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.009+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fb401f030 con 0x7f4fb81085e0 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.207+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4fb401b440 con 0x7f4fb81085e0 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.210+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f4fb80640a0 con 0x7f4fb81085e0 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.210+0000 7f4faeffd700 1 -- 172.21.15.96:0/1405667539 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f4fb402a300 con 0x7f4fb81085e0 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4fa40395b0 msgr2=0x7f4fa403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4fa40395b0 0x7f4fa403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8006fd0 tx=0x7f4fa8006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 msgr2=0x7f4fb819c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb819c4e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40119f0 tx=0x7f4fb4011ad0 comp rx=0 tx=0).stop 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 shutdown_connections 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4fa40395b0 0x7f4fa403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 --2- 172.21.15.96:0/1405667539 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4fb81085e0 0x7f4fb819c4e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 >> 172.21.15.96:0/1405667539 conn(0x7f4fb8103ba0 msgr2=0x7f4fb8104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 shutdown_connections 2024-04-25T20:47:16.008 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:15.213+0000 7f4fbfb43700 1 -- 172.21.15.96:0/1405667539 wait complete. 2024-04-25T20:47:16.011 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-25T20:47:16.011 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout disabled 2024-04-25T20:47:16.018 INFO:teuthology.orchestra.run.smithi096.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-25T20:47:16.018 INFO:teuthology.orchestra.run.smithi096.stdout:systemctl: stdout inactive 2024-04-25T20:47:16.018 INFO:teuthology.orchestra.run.smithi096.stdout:firewalld.service is not enabled 2024-04-25T20:47:16.018 INFO:teuthology.orchestra.run.smithi096.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout:Ceph Dashboard is now available at: 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: URL: https://smithi096:8443/ 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: User: admin 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: Password: poki16kiz3 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:16.020 INFO:teuthology.orchestra.run.smithi096.stdout:Saving cluster configuration to /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config directory 2024-04-25T20:47:16.021 INFO:teuthology.orchestra.run.smithi096.stdout:Enabling autotune for osd_memory_target 2024-04-25T20:47:16.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:16 smithi096 ceph-mon[29288]: mgrmap e13: x(active, since 6s) 2024-04-25T20:47:16.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:16 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1405667539' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10e134f700 1 Processor -- start 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10e134f700 1 -- start start 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10e134f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc10b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10e134f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc07ce70 con 0x7f10dc1090c0 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10daffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc10b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10daffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc10b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37122/0 (socket says 172.21.15.96:37122) 2024-04-25T20:47:17.236 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10daffd700 1 -- 172.21.15.96:0/345390529 learned_addr learned my addr 172.21.15.96:0/345390529 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.451+0000 7f10daffd700 1 -- 172.21.15.96:0/345390529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc10b9f0 con 0x7f10dc1090c0 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10daffd700 1 --2- 172.21.15.96:0/345390529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc10b4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098a0 tx=0x7f10cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80860b4dbe8a3c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10d9ffb700 1 -- 172.21.15.96:0/345390529 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc004160 con 0x7f10dc1090c0 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10d9ffb700 1 -- 172.21.15.96:0/345390529 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc004300 con 0x7f10dc1090c0 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10d9ffb700 1 -- 172.21.15.96:0/345390529 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc011a20 con 0x7f10dc1090c0 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10e134f700 1 -- 172.21.15.96:0/345390529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 msgr2=0x7f10dc10b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10e134f700 1 --2- 172.21.15.96:0/345390529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc10b4b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098a0 tx=0x7f10cc009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10e134f700 1 -- 172.21.15.96:0/345390529 shutdown_connections 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10e134f700 1 --2- 172.21.15.96:0/345390529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc10b4b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10e134f700 1 -- 172.21.15.96:0/345390529 >> 172.21.15.96:0/345390529 conn(0x7f10dc102ad0 msgr2=0x7f10dc104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.452+0000 7f10e134f700 1 -- 172.21.15.96:0/345390529 shutdown_connections 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.453+0000 7f10e134f700 1 -- 172.21.15.96:0/345390529 wait complete. 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.453+0000 7f10e134f700 1 Processor -- start 2024-04-25T20:47:17.237 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.453+0000 7f10e134f700 1 -- start start 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.453+0000 7f10e134f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc108940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.453+0000 7f10e134f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc10c9f0 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.453+0000 7f10daffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc108940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10daffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc108940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37130/0 (socket says 172.21.15.96:37130) 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10daffd700 1 -- 172.21.15.96:0/842038535 learned_addr learned my addr 172.21.15.96:0/842038535 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10daffd700 1 -- 172.21.15.96:0/842038535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc108e80 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10daffd700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc108940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0116e0 tx=0x7f10cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc003b20 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc107070 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10cc01b8f0 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.454+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10cc0238c0 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.455+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc1075b0 con 0x7f10dc1090c0 2024-04-25T20:47:17.238 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.455+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f10cc023b70 con 0x7f10dc1090c0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.455+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc0640a0 con 0x7f10dc1090c0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.455+0000 7f10c3fff700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f10c4039650 0x7f10c403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.455+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f10cc04e360 con 0x7f10dc1090c0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.456+0000 7f10da7fc700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f10c4039650 0x7f10c403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.456+0000 7f10da7fc700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f10c4039650 0x7f10c403bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006fd0 tx=0x7f10d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.460+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10cc01f030 con 0x7f10dc1090c0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.656+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f10dc107830 con 0x7f10dc1090c0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.657+0000 7f10c3fff700 1 -- 172.21.15.96:0/842038535 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f10cc018590 con 0x7f10dc1090c0 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f10c4039650 msgr2=0x7f10c403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f10c4039650 0x7f10c403bb10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f10d0006fd0 tx=0x7f10d0006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 msgr2=0x7f10dc108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc108940 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0116e0 tx=0x7f10cc011aa0 comp rx=0 tx=0).stop 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 shutdown_connections 2024-04-25T20:47:17.239 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f10c4039650 0x7f10c403bb10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:17.240 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 --2- 172.21.15.96:0/842038535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10dc1090c0 0x7f10dc108940 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:17.240 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 >> 172.21.15.96:0/842038535 conn(0x7f10dc102ad0 msgr2=0x7f10dc103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:17.240 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 shutdown_connections 2024-04-25T20:47:17.240 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:16.660+0000 7f10e134f700 1 -- 172.21.15.96:0/842038535 wait complete. 2024-04-25T20:47:18.567 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f559a187700 1 Processor -- start 2024-04-25T20:47:18.567 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f559a187700 1 -- start start 2024-04-25T20:47:18.567 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f559a187700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f5594107930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:18.567 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f559a187700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5594107f00 con 0x7f5594107510 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f55937fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f5594107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f55937fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f5594107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37138/0 (socket says 172.21.15.96:37138) 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f55937fe700 1 -- 172.21.15.96:0/229697221 learned_addr learned my addr 172.21.15.96:0/229697221 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.658+0000 7f55937fe700 1 -- 172.21.15.96:0/229697221 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5594108710 con 0x7f5594107510 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f55937fe700 1 --2- 172.21.15.96:0/229697221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f5594107930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f557c0098a0 tx=0x7f557c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aab2193e7df074 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f55927fc700 1 -- 172.21.15.96:0/229697221 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557c004160 con 0x7f5594107510 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f55927fc700 1 -- 172.21.15.96:0/229697221 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c004300 con 0x7f5594107510 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f55927fc700 1 -- 172.21.15.96:0/229697221 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557c011a20 con 0x7f5594107510 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 -- 172.21.15.96:0/229697221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 msgr2=0x7f5594107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 --2- 172.21.15.96:0/229697221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f5594107930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f557c0098a0 tx=0x7f557c009bb0 comp rx=0 tx=0).stop 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 -- 172.21.15.96:0/229697221 shutdown_connections 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 --2- 172.21.15.96:0/229697221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f5594107930 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:18.568 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 -- 172.21.15.96:0/229697221 >> 172.21.15.96:0/229697221 conn(0x7f5594102ad0 msgr2=0x7f5594104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 -- 172.21.15.96:0/229697221 shutdown_connections 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.659+0000 7f559a187700 1 -- 172.21.15.96:0/229697221 wait complete. 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.660+0000 7f559a187700 1 Processor -- start 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.660+0000 7f559a187700 1 -- start start 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.660+0000 7f559a187700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f559419c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.660+0000 7f559a187700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f559410bac0 con 0x7f5594107510 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.660+0000 7f55937fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f559419c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f55937fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f559419c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37144/0 (socket says 172.21.15.96:37144) 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f55937fe700 1 -- 172.21.15.96:0/3249979799 learned_addr learned my addr 172.21.15.96:0/3249979799 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f55937fe700 1 -- 172.21.15.96:0/3249979799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559419ca10 con 0x7f5594107510 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f55937fe700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f559419c4d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f557c0119f0 tx=0x7f557c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557c0039e0 con 0x7f5594107510 2024-04-25T20:47:18.569 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f557c01b8f0 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559419cca0 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557c0237a0 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.661+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559419f990 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.662+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f557c023aa0 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.662+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559410bc40 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.662+0000 7f5590ff9700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5580039650 0x7f558003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.662+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f557c04e290 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.662+0000 7f5592ffd700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5580039650 0x7f558003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.663+0000 7f5592ffd700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5580039650 0x7f558003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5584006fd0 tx=0x7f5584006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.667+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f557c01f030 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.974+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f559402e1c0 con 0x7f5594107510 2024-04-25T20:47:18.570 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.976+0000 7f5590ff9700 1 -- 172.21.15.96:0/3249979799 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f557c01bb10 con 0x7f5594107510 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5580039650 msgr2=0x7f558003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5580039650 0x7f558003bb10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5584006fd0 tx=0x7f5584006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 msgr2=0x7f559419c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f559419c4d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f557c0119f0 tx=0x7f557c011ad0 comp rx=0 tx=0).stop 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 shutdown_connections 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5580039650 0x7f558003bb10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 --2- 172.21.15.96:0/3249979799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5594107510 0x7f559419c4d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 >> 172.21.15.96:0/3249979799 conn(0x7f5594102ad0 msgr2=0x7f5594103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 shutdown_connections 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr 2024-04-25T20:47:17.979+0000 7f559a187700 1 -- 172.21.15.96:0/3249979799 wait complete. 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.571 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:Or, if you are only running a single cluster on this host: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: ceph telemetry on 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:For more information see: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:18.572 INFO:teuthology.orchestra.run.smithi096.stdout:Bootstrap complete. 2024-04-25T20:47:18.604 INFO:tasks.cephadm:Fetching config... 2024-04-25T20:47:18.604 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:47:18.604 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-25T20:47:18.619 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-25T20:47:18.619 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:47:18.619 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-25T20:47:18.674 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-25T20:47:18.674 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:47:18.674 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-25T20:47:18.737 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-25T20:47:18.737 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:47:18.737 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-25T20:47:18.793 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-25T20:47:18.793 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:47:18.864 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:47:18.875 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:47:18.914 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:47:18.926 DEBUG:teuthology.orchestra.run.smithi139:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-25T20:47:18.965 INFO:teuthology.orchestra.run.smithi139.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDmBxNgMOqGb7m8MV+Y7UPo7hi3NeJ1gz7R1T5xHjz9yic7jXWauNWo4GCZqVOMQ39L2toIEd4YHoOBp6BC38EeL4OG8XMzhF4lhdlopyxupy4JgnziRJBE1crdlTy++OIhMHkmwpuXNcm6JYJnQNZ49FUYRhnfsZA7I56F78GqLfdak/Zyvdia7lBbCwdQkkue9xx66tPjneLwl5kVEZJ+13jI3I/vbrVK29cHBtOo4QlYw5acgYhf6/QzTTQVUoHGsj5Sbvcqiw9aGuoNdtPG9pMVQjXKOwejBPwcqFluptFVUY7DFJE5bxWO7pL8MoxfSB7hgQBaEjb/pAYrdVmOJgX9zAPU7WJnldxKm4sYzgYb4adYuyy4G11YYT+6AZHLWUtbozXaubxp6RqvlpfXPyZNs7XquIatqbkx/KhDhzgdEPD+mw7jHz9cVR+fVqqXftVB0ttFd0ko2qiqTHr1Jn2+EwD3vgUsrbQ/Yhx8eGisQs6FsQ8ETTsW5ANNMh0= ceph-b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:47:18.977 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-25T20:47:18.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:18 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3249979799' entity='client.admin' 2024-04-25T20:47:19.216 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.650+0000 7f41277cc700 1 -- 172.21.15.96:0/2111456193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 msgr2=0x7f41201017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.650+0000 7f41277cc700 1 --2- 172.21.15.96:0/2111456193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41201017c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f411c009910 tx=0x7f411c009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.650+0000 7f41277cc700 1 -- 172.21.15.96:0/2111456193 shutdown_connections 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.650+0000 7f41277cc700 1 --2- 172.21.15.96:0/2111456193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41201017c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.650+0000 7f41277cc700 1 -- 172.21.15.96:0/2111456193 >> 172.21.15.96:0/2111456193 conn(0x7f41200fc940 msgr2=0x7f41200fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f41277cc700 1 -- 172.21.15.96:0/2111456193 shutdown_connections 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f41277cc700 1 -- 172.21.15.96:0/2111456193 wait complete. 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f41277cc700 1 Processor -- start 2024-04-25T20:47:21.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f41277cc700 1 -- start start 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f41277cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41200fe800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f41277cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41201072e0 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f4125568700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41200fe800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f4125568700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41200fe800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37166/0 (socket says 172.21.15.96:37166) 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.651+0000 7f4125568700 1 -- 172.21.15.96:0/1601265386 learned_addr learned my addr 172.21.15.96:0/1601265386 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f4125568700 1 -- 172.21.15.96:0/1601265386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41200fed40 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f4125568700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41200fe800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f411c005800 tx=0x7f411c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c027070 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c00be10 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41200fcf30 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c00f460 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.652+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41200fd470 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.653+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f411c02a070 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.653+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4120107a60 con 0x7f41201013a0 2024-04-25T20:47:21.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.653+0000 7f41167fc700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f410c039650 0x7f410c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:21.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.653+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f411c01e030 con 0x7f41201013a0 2024-04-25T20:47:21.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.653+0000 7f4124d67700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f410c039650 0x7f410c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:21.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.654+0000 7f4124d67700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f410c039650 0x7f410c03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4110006fd0 tx=0x7f4110006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:21.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.658+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f411c013070 con 0x7f41201013a0 2024-04-25T20:47:21.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.864+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4120191010 con 0x7f41201013a0 2024-04-25T20:47:21.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.867+0000 7f41167fc700 1 -- 172.21.15.96:0/1601265386 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f411c031020 con 0x7f41201013a0 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.869+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f410c039650 msgr2=0x7f410c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.869+0000 7f41277cc700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f410c039650 0x7f410c03bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4110006fd0 tx=0x7f4110006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.869+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 msgr2=0x7f41200fe800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.869+0000 7f41277cc700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41200fe800 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f411c005800 tx=0x7f411c009f60 comp rx=0 tx=0).stop 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.870+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 shutdown_connections 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.870+0000 7f41277cc700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f410c039650 0x7f410c03bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.870+0000 7f41277cc700 1 --2- 172.21.15.96:0/1601265386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41201013a0 0x7f41200fe800 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.870+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 >> 172.21.15.96:0/1601265386 conn(0x7f41200fc940 msgr2=0x7f412006a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.870+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 shutdown_connections 2024-04-25T20:47:21.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:21.870+0000 7f41277cc700 1 -- 172.21.15.96:0/1601265386 wait complete. 2024-04-25T20:47:22.468 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-25T20:47:22.468 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-25T20:47:23.079 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:23.202 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:22 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1601265386' entity='client.admin' 2024-04-25T20:47:24.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:24.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:24.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:47:24.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:24.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:24.084 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:24.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3683421737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 msgr2=0x7f4d88105340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:24.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 --2- 172.21.15.96:0/3683421737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88105340 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4d7800b0a0 tx=0x7f4d7800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:47:24.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3683421737 shutdown_connections 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 --2- 172.21.15.96:0/3683421737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88105340 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3683421737 >> 172.21.15.96:0/3683421737 conn(0x7f4d880fc980 msgr2=0x7f4d880feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3683421737 shutdown_connections 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3683421737 wait complete. 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 Processor -- start 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.534+0000 7f4d8f9fc700 1 -- start start 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8f9fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8f9fc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881083a0 con 0x7f4d88102f50 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8d798700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8d798700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37176/0 (socket says 172.21.15.96:37176) 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8d798700 1 -- 172.21.15.96:0/3833435248 learned_addr learned my addr 172.21.15.96:0/3833435248 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8d798700 1 -- 172.21.15.96:0/3833435248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d88197ff0 con 0x7f4d88102f50 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8d798700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88197ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007730 tx=0x7f4d78003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d78021020 con 0x7f4d88102f50 2024-04-25T20:47:24.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d7800bdd0 con 0x7f4d88102f50 2024-04-25T20:47:24.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d88198280 con 0x7f4d88102f50 2024-04-25T20:47:24.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4d78017040 con 0x7f4d88102f50 2024-04-25T20:47:24.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.535+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d8819af20 con 0x7f4d88102f50 2024-04-25T20:47:24.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.536+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4d78024020 con 0x7f4d88102f50 2024-04-25T20:47:24.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.536+0000 7f4d867fc700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4d700396b0 0x7f4d7003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:24.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.536+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d88198740 con 0x7f4d88102f50 2024-04-25T20:47:24.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.536+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4d7800f050 con 0x7f4d88102f50 2024-04-25T20:47:24.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.536+0000 7f4d8cf97700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4d700396b0 0x7f4d7003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:24.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.537+0000 7f4d8cf97700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4d700396b0 0x7f4d7003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c006fd0 tx=0x7f4d7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:24.541 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.541+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4d7802b020 con 0x7f4d88102f50 2024-04-25T20:47:24.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.742+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f4d8802dff0 con 0x7f4d700396b0 2024-04-25T20:47:24.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.746+0000 7f4d867fc700 1 -- 172.21.15.96:0/3833435248 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4d8802dff0 con 0x7f4d700396b0 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4d700396b0 msgr2=0x7f4d7003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4d700396b0 0x7f4d7003bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c006fd0 tx=0x7f4d7c006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 msgr2=0x7f4d88197ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88197ab0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4d78007730 tx=0x7f4d78003ce0 comp rx=0 tx=0).stop 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 shutdown_connections 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4d700396b0 0x7f4d7003bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 --2- 172.21.15.96:0/3833435248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d88102f50 0x7f4d88197ab0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 >> 172.21.15.96:0/3833435248 conn(0x7f4d880fc980 msgr2=0x7f4d880fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:24.750 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 shutdown_connections 2024-04-25T20:47:24.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:24.748+0000 7f4d8f9fc700 1 -- 172.21.15.96:0/3833435248 wait complete. 2024-04-25T20:47:25.507 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2024-04-25T20:47:25.507 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:47:25.507 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2024-04-25T20:47:25.526 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:47:25.526 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:25.583 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2024-04-25T20:47:25.583 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch host add smithi097 2024-04-25T20:47:25.818 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:25.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:25.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:26.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:26.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:26 smithi096 ceph-mon[29288]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:47:27.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 -- 172.21.15.96:0/2559071973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 msgr2=0x7f013c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 --2- 172.21.15.96:0/2559071973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c1088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0130009910 tx=0x7f0130009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 -- 172.21.15.96:0/2559071973 shutdown_connections 2024-04-25T20:47:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 --2- 172.21.15.96:0/2559071973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c1088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 -- 172.21.15.96:0/2559071973 >> 172.21.15.96:0/2559071973 conn(0x7f013c0fdf60 msgr2=0x7f013c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 -- 172.21.15.96:0/2559071973 shutdown_connections 2024-04-25T20:47:27.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.338+0000 7f0142e3d700 1 -- 172.21.15.96:0/2559071973 wait complete. 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0142e3d700 1 Processor -- start 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0142e3d700 1 -- start start 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0142e3d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0142e3d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f013c10c940 con 0x7f013c1084e0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0140bd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0140bd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35336/0 (socket says 172.21.15.96:35336) 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.339+0000 7f0140bd9700 1 -- 172.21.15.96:0/3032522671 learned_addr learned my addr 172.21.15.96:0/3032522671 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0140bd9700 1 -- 172.21.15.96:0/3032522671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f013c1983d0 con 0x7f013c1084e0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0140bd9700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c197e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0130006010 tx=0x7f0130005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0130026070 con 0x7f013c1084e0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f013000bba0 con 0x7f013c1084e0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f013c1986c0 con 0x7f013c1084e0 2024-04-25T20:47:27.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0130003910 con 0x7f013c1084e0 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.340+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f013c19c760 con 0x7f013c1084e0 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.341+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f0130029070 con 0x7f013c1084e0 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.341+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f013c103b90 con 0x7f013c1084e0 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.341+0000 7f0139ffb700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f012c039650 0x7f012c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.341+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f013001e070 con 0x7f013c1084e0 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.341+0000 7f013bfff700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f012c039650 0x7f012c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:27.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.342+0000 7f013bfff700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f012c039650 0x7f012c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0128006fd0 tx=0x7f0128006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:27.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.346+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0130013070 con 0x7f013c1084e0 2024-04-25T20:47:27.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:27.546+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) -- 0x7f013c1991f0 con 0x7f012c039650 2024-04-25T20:47:28.653 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:28 smithi096 ceph-mon[29288]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:28.653 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:28 smithi096 ceph-mon[29288]: Deploying cephadm binary to smithi097 2024-04-25T20:47:29.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:29 smithi096 ceph-mon[29288]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:31.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.639+0000 7f0139ffb700 1 -- 172.21.15.96:0/3032522671 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f013c1991f0 con 0x7f012c039650 2024-04-25T20:47:31.640 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi097' with addr '172.21.15.97' 2024-04-25T20:47:31.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f012c039650 msgr2=0x7f012c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:31.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f012c039650 0x7f012c03bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0128006fd0 tx=0x7f0128006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:31.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 msgr2=0x7f013c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:31.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c197e90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0130006010 tx=0x7f0130005780 comp rx=0 tx=0).stop 2024-04-25T20:47:31.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 shutdown_connections 2024-04-25T20:47:31.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f012c039650 0x7f012c03bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:31.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 --2- 172.21.15.96:0/3032522671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f013c1084e0 0x7f013c197e90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:31.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 >> 172.21.15.96:0/3032522671 conn(0x7f013c0fdf60 msgr2=0x7f013c109ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:31.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 shutdown_connections 2024-04-25T20:47:31.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:31.641+0000 7f0142e3d700 1 -- 172.21.15.96:0/3032522671 wait complete. 2024-04-25T20:47:32.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:32 smithi096 ceph-mon[29288]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:32.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:32.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:32.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:32.295 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T20:47:32.531 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:33.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:33 smithi096 ceph-mon[29288]: Added host smithi097 2024-04-25T20:47:33.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:34.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 -- 172.21.15.96:0/1698128164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 msgr2=0x7fef901019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 --2- 172.21.15.96:0/1698128164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef901019d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fef80009910 tx=0x7fef80009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 -- 172.21.15.96:0/1698128164 shutdown_connections 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 --2- 172.21.15.96:0/1698128164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef901019d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 -- 172.21.15.96:0/1698128164 >> 172.21.15.96:0/1698128164 conn(0x7fef90076ea0 msgr2=0x7fef900772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 -- 172.21.15.96:0/1698128164 shutdown_connections 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.088+0000 7fef987fe700 1 -- 172.21.15.96:0/1698128164 wait complete. 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef987fe700 1 Processor -- start 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef987fe700 1 -- start start 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef987fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef90197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef987fe700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef9010b820 con 0x7fef901015f0 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef9659a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef90197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:34.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef9659a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef90197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35358/0 (socket says 172.21.15.96:35358) 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef9659a700 1 -- 172.21.15.96:0/2621786038 learned_addr learned my addr 172.21.15.96:0/2621786038 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.089+0000 7fef9659a700 1 -- 172.21.15.96:0/2621786038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef90198420 con 0x7fef901015f0 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.090+0000 7fef9659a700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef90197ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef800056d0 tx=0x7fef800057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.090+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef80026070 con 0x7fef901015f0 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.090+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef90198710 con 0x7fef901015f0 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.090+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8000bba0 con 0x7fef901015f0 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.090+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fef80003920 con 0x7fef901015f0 2024-04-25T20:47:34.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.090+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9019c760 con 0x7fef901015f0 2024-04-25T20:47:34.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.091+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fef80029070 con 0x7fef901015f0 2024-04-25T20:47:34.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.091+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef90102a80 con 0x7fef901015f0 2024-04-25T20:47:34.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.091+0000 7fef877fe700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef7c039650 0x7fef7c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:34.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.091+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fef8001e070 con 0x7fef901015f0 2024-04-25T20:47:34.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.091+0000 7fef95d99700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef7c039650 0x7fef7c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:34.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.091+0000 7fef95d99700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef7c039650 0x7fef7c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef8c006fd0 tx=0x7fef8c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.096+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fef80013070 con 0x7fef901015f0 2024-04-25T20:47:34.300 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.299+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fef901991f0 con 0x7fef7c039650 2024-04-25T20:47:34.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.300+0000 7fef877fe700 1 -- 172.21.15.96:0/2621786038 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fef901991f0 con 0x7fef7c039650 2024-04-25T20:47:34.301 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:34.301 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2024-04-25T20:47:34.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef7c039650 msgr2=0x7fef7c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:34.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef7c039650 0x7fef7c03bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fef8c006fd0 tx=0x7fef8c006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:34.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 msgr2=0x7fef90197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef90197ee0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fef800056d0 tx=0x7fef800057b0 comp rx=0 tx=0).stop 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 shutdown_connections 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef7c039650 0x7fef7c03bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 --2- 172.21.15.96:0/2621786038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef901015f0 0x7fef90197ee0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.302+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 >> 172.21.15.96:0/2621786038 conn(0x7fef90076ea0 msgr2=0x7fef901091b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.303+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 shutdown_connections 2024-04-25T20:47:34.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:34.303+0000 7fef987fe700 1 -- 172.21.15.96:0/2621786038 wait complete. 2024-04-25T20:47:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:34 smithi096 ceph-mon[29288]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:34.890 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi139 2024-04-25T20:47:34.890 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:47:34.890 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.conf 2024-04-25T20:47:34.909 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:47:34.909 DEBUG:teuthology.orchestra.run.smithi139:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:34.966 INFO:tasks.cephadm:Adding host smithi139 to orchestrator... 2024-04-25T20:47:34.967 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch host add smithi139 2024-04-25T20:47:35.201 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:35.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:35 smithi096 ceph-mon[29288]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:47:35.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:36.481 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:36 smithi096 ceph-mon[29288]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 -- 172.21.15.96:0/2114913563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 msgr2=0x7feb1c108930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 --2- 172.21.15.96:0/2114913563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c108930 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7feb18009910 tx=0x7feb18009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 -- 172.21.15.96:0/2114913563 shutdown_connections 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 --2- 172.21.15.96:0/2114913563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c108930 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 -- 172.21.15.96:0/2114913563 >> 172.21.15.96:0/2114913563 conn(0x7feb1c077ac0 msgr2=0x7feb1c077ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 -- 172.21.15.96:0/2114913563 shutdown_connections 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 -- 172.21.15.96:0/2114913563 wait complete. 2024-04-25T20:47:36.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.725+0000 7feb240ee700 1 Processor -- start 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb240ee700 1 -- start start 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb240ee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c101bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb240ee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb1c10c9c0 con 0x7feb1c108550 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb21e8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c101bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb21e8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c101bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46066/0 (socket says 172.21.15.96:46066) 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb21e8a700 1 -- 172.21.15.96:0/1972345056 learned_addr learned my addr 172.21.15.96:0/1972345056 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb21e8a700 1 -- 172.21.15.96:0/1972345056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb1c103aa0 con 0x7feb1c108550 2024-04-25T20:47:36.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb21e8a700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c101bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb18006010 tx=0x7feb18005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:36.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb18026070 con 0x7feb1c108550 2024-04-25T20:47:36.729 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb1800bc00 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb1c1020f0 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7feb18003910 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.726+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb1c1025b0 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.727+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7feb18029070 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.727+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb1c0640a0 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.727+0000 7feb12ffd700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb080396b0 0x7feb0803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.728+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7feb1801e070 con 0x7feb1c108550 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.728+0000 7feb21689700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb080396b0 0x7feb0803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:36.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.728+0000 7feb21689700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb080396b0 0x7feb0803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb0c006fd0 tx=0x7feb0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:36.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.732+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb18013070 con 0x7feb1c108550 2024-04-25T20:47:36.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:36.934+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}) -- 0x7feb1c1027e0 con 0x7feb080396b0 2024-04-25T20:47:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:39.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:39 smithi096 ceph-mon[29288]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:47:39.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:39 smithi096 ceph-mon[29288]: Deploying cephadm binary to smithi139 2024-04-25T20:47:39.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:39 smithi096 ceph-mon[29288]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:39.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:39 smithi096 ceph-mon[29288]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:40.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:40 smithi096 ceph-mon[29288]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:41.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.052+0000 7feb12ffd700 1 -- 172.21.15.96:0/1972345056 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7feb1c1027e0 con 0x7feb080396b0 2024-04-25T20:47:41.053 INFO:teuthology.orchestra.run.smithi096.stdout:Added host 'smithi139' with addr '172.21.15.139' 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb080396b0 msgr2=0x7feb0803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb080396b0 0x7feb0803bb70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7feb0c006fd0 tx=0x7feb0c006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 msgr2=0x7feb1c101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c101bb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feb18006010 tx=0x7feb18005780 comp rx=0 tx=0).stop 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 shutdown_connections 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb080396b0 0x7feb0803bb70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 --2- 172.21.15.96:0/1972345056 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb1c108550 0x7feb1c101bb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 >> 172.21.15.96:0/1972345056 conn(0x7feb1c077ac0 msgr2=0x7feb1c0fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:41.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 shutdown_connections 2024-04-25T20:47:41.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:41.054+0000 7feb240ee700 1 -- 172.21.15.96:0/1972345056 wait complete. 2024-04-25T20:47:41.670 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: Added host smithi139 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:41.671 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:41 smithi096 ceph-mon[29288]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:41.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-25T20:47:41.926 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:42.851 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:42.851 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:43.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 -- 172.21.15.96:0/975439086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 msgr2=0x7f5bd81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:43.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 --2- 172.21.15.96:0/975439086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd81088c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009910 tx=0x7f5bc8009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:43.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 -- 172.21.15.96:0/975439086 shutdown_connections 2024-04-25T20:47:43.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 --2- 172.21.15.96:0/975439086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd81088c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:43.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 -- 172.21.15.96:0/975439086 >> 172.21.15.96:0/975439086 conn(0x7f5bd8076ea0 msgr2=0x7f5bd80772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:43.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 -- 172.21.15.96:0/975439086 shutdown_connections 2024-04-25T20:47:43.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.607+0000 7f5bdce6d700 1 -- 172.21.15.96:0/975439086 wait complete. 2024-04-25T20:47:43.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bdce6d700 1 Processor -- start 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bdce6d700 1 -- start start 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bdce6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bdce6d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd810c920 con 0x7f5bd81084e0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bd659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bd659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd8197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46082/0 (socket says 172.21.15.96:46082) 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bd659c700 1 -- 172.21.15.96:0/681109263 learned_addr learned my addr 172.21.15.96:0/681109263 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.608+0000 7f5bd659c700 1 -- 172.21.15.96:0/681109263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd81983f0 con 0x7f5bd81084e0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.609+0000 7f5bd659c700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd8197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8006010 tx=0x7f5bc8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.609+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bc8026070 con 0x7f5bd81084e0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.609+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5bc800bba0 con 0x7f5bd81084e0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.609+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd81986e0 con 0x7f5bd81084e0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.609+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5bc8003910 con 0x7f5bd81084e0 2024-04-25T20:47:43.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.609+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd819c780 con 0x7f5bd81084e0 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.610+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5bc8029070 con 0x7f5bd81084e0 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.610+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd8103b90 con 0x7f5bd81084e0 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.610+0000 7f5bcffff700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5bc4039650 0x7f5bc403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.610+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5bc801e070 con 0x7f5bd81084e0 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.610+0000 7f5bcdd9b700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5bc4039650 0x7f5bc403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.610+0000 7f5bcdd9b700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5bc4039650 0x7f5bc403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0006fd0 tx=0x7f5bc0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:43.617 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.615+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5bc8013070 con 0x7f5bd81084e0 2024-04-25T20:47:43.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:43 smithi096 ceph-mon[29288]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:43.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.818+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bd8199210 con 0x7f5bc4039650 2024-04-25T20:47:43.820 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.819+0000 7f5bcffff700 1 -- 172.21.15.96:0/681109263 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f5bd8199210 con 0x7f5bc4039650 2024-04-25T20:47:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:47:43.821 INFO:teuthology.orchestra.run.smithi096.stdout:[{"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.139", "hostname": "smithi139", "labels": [], "status": ""}] 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5bc4039650 msgr2=0x7f5bc403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5bc4039650 0x7f5bc403bb10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0006fd0 tx=0x7f5bc0006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 msgr2=0x7f5bd8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd8197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8006010 tx=0x7f5bc8005780 comp rx=0 tx=0).stop 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 shutdown_connections 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5bc4039650 0x7f5bc403bb10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 --2- 172.21.15.96:0/681109263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bd81084e0 0x7f5bd8197eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 >> 172.21.15.96:0/681109263 conn(0x7f5bd8076ea0 msgr2=0x7f5bd80fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 shutdown_connections 2024-04-25T20:47:43.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:43.822+0000 7f5bdce6d700 1 -- 172.21.15.96:0/681109263 wait complete. 2024-04-25T20:47:44.427 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-25T20:47:44.427 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-25T20:47:44.658 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:47:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:44 smithi096 ceph-mon[29288]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:47:45.975 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:45 smithi096 ceph-mon[29288]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 -- 172.21.15.96:0/748989239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 msgr2=0x7f880c1088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 --2- 172.21.15.96:0/748989239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c1088c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f87fc009910 tx=0x7f87fc009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 -- 172.21.15.96:0/748989239 shutdown_connections 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 --2- 172.21.15.96:0/748989239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c1088c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 -- 172.21.15.96:0/748989239 >> 172.21.15.96:0/748989239 conn(0x7f880c076ea0 msgr2=0x7f880c0772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 -- 172.21.15.96:0/748989239 shutdown_connections 2024-04-25T20:47:46.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.208+0000 7f881117a700 1 -- 172.21.15.96:0/748989239 wait complete. 2024-04-25T20:47:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f881117a700 1 Processor -- start 2024-04-25T20:47:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f881117a700 1 -- start start 2024-04-25T20:47:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f881117a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f881117a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f880c10c920 con 0x7f880c1084e0 2024-04-25T20:47:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f880ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:46.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f880ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46102/0 (socket says 172.21.15.96:46102) 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.209+0000 7f880ad9d700 1 -- 172.21.15.96:0/1453273568 learned_addr learned my addr 172.21.15.96:0/1453273568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f880ad9d700 1 -- 172.21.15.96:0/1453273568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c103350 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f880ad9d700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c101460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0056d0 tx=0x7f87fc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87fc026070 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87fc00bc00 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c101a00 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87fc003920 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.210+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c101ec0 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.211+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f87fc029070 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.211+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c103b90 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.211+0000 7f8808d99700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f87f8039650 0x7f87f803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.211+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f87fc01e070 con 0x7f880c1084e0 2024-04-25T20:47:46.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.211+0000 7f880259c700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f87f8039650 0x7f87f803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:46.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.212+0000 7f880259c700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f87f8039650 0x7f87f803bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006fd0 tx=0x7f87f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:46.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.216+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f87fc013070 con 0x7f880c1084e0 2024-04-25T20:47:46.422 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.421+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f880c102990 con 0x7f880c1084e0 2024-04-25T20:47:46.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.642+0000 7f8808d99700 1 -- 172.21.15.96:0/1453273568 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f87fc030090 con 0x7f880c1084e0 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f87f8039650 msgr2=0x7f87f803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f87f8039650 0x7f87f803bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f87f4006fd0 tx=0x7f87f4006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 msgr2=0x7f880c101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c101460 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f87fc0056d0 tx=0x7f87fc0057b0 comp rx=0 tx=0).stop 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 shutdown_connections 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f87f8039650 0x7f87f803bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 --2- 172.21.15.96:0/1453273568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f880c1084e0 0x7f880c101460 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:46.646 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 >> 172.21.15.96:0/1453273568 conn(0x7f880c076ea0 msgr2=0x7f880c0fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:46.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 shutdown_connections 2024-04-25T20:47:46.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:47:46.645+0000 7f881117a700 1 -- 172.21.15.96:0/1453273568 wait complete. 2024-04-25T20:47:46.647 INFO:teuthology.orchestra.run.smithi096.stderr:adjusted tunables profile to default 2024-04-25T20:47:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:46 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1453273568' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:47:47.350 INFO:tasks.cephadm:Adding mon.a on smithi096 2024-04-25T20:47:47.350 INFO:tasks.cephadm:Adding mon.b on smithi097 2024-04-25T20:47:47.350 INFO:tasks.cephadm:Adding mon.c on smithi139 2024-04-25T20:47:47.351 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c' 2024-04-25T20:47:47.584 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:47.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1453273568' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:48.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:47 smithi096 ceph-mon[29288]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 -- 172.21.15.139:0/3914919403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 msgr2=0x7f80f01088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 --2- 172.21.15.139:0/3914919403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f01088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f80e0009910 tx=0x7f80e0009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 -- 172.21.15.139:0/3914919403 shutdown_connections 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 --2- 172.21.15.139:0/3914919403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f01088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 -- 172.21.15.139:0/3914919403 >> 172.21.15.139:0/3914919403 conn(0x7f80f00fdf80 msgr2=0x7f80f01003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 -- 172.21.15.139:0/3914919403 shutdown_connections 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.135+0000 7f80f6594700 1 -- 172.21.15.139:0/3914919403 wait complete. 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80f6594700 1 Processor -- start 2024-04-25T20:47:49.137 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80f6594700 1 -- start start 2024-04-25T20:47:49.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80f6594700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f0197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:49.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80f6594700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f80f010c940 con 0x7f80f01084e0 2024-04-25T20:47:49.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80effff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f0197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:49.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80effff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f0197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.139:42132/0 (socket says 172.21.15.139:42132) 2024-04-25T20:47:49.140 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.136+0000 7f80effff700 1 -- 172.21.15.139:0/2506476730 learned_addr learned my addr 172.21.15.139:0/2506476730 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80effff700 1 -- 172.21.15.139:0/2506476730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f01983f0 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80effff700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f0197eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80e00056d0 tx=0x7f80e00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e0026070 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f01986e0 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f80e000bba0 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f80e0003920 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.137+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f019c780 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.138+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f80e0029070 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.138+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80f0103b90 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.138+0000 7f80ed7fa700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f80d8039650 0x7f80d803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.138+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f80e001e070 con 0x7f80f01084e0 2024-04-25T20:47:49.141 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.138+0000 7f80ef7fe700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f80d8039650 0x7f80d803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:49.142 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.139+0000 7f80ef7fe700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f80d8039650 0x7f80d803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80e4006fd0 tx=0x7f80e4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:49.144 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.144+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f80e0013070 con 0x7f80f01084e0 2024-04-25T20:47:49.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:48 smithi096 ceph-mon[29288]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:49.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:48 smithi096 ceph-mon[29288]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:47:49.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:49.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:49.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:49.352 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.351+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c", "target": ["mon-mgr", ""]}) -- 0x7f80f0199210 con 0x7f80d8039650 2024-04-25T20:47:49.358 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.357+0000 7f80ed7fa700 1 -- 172.21.15.139:0/2506476730 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f80f0199210 con 0x7f80d8039650 2024-04-25T20:47:49.358 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mon update... 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f80d8039650 msgr2=0x7f80d803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f80d8039650 0x7f80d803bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f80e4006fd0 tx=0x7f80e4006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 msgr2=0x7f80f0197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f0197eb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f80e00056d0 tx=0x7f80e00057b0 comp rx=0 tx=0).stop 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 shutdown_connections 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f80d8039650 0x7f80d803bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 --2- 172.21.15.139:0/2506476730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f80f01084e0 0x7f80f0197eb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 >> 172.21.15.139:0/2506476730 conn(0x7f80f00fdf80 msgr2=0x7f80f010a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:49.361 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 shutdown_connections 2024-04-25T20:47:49.362 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:49.360+0000 7f80f6594700 1 -- 172.21.15.139:0/2506476730 wait complete. 2024-04-25T20:47:49.998 DEBUG:teuthology.orchestra.run.smithi097:mon.b> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.b.service 2024-04-25T20:47:50.001 DEBUG:teuthology.orchestra.run.smithi139:mon.c> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.c.service 2024-04-25T20:47:50.003 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:47:50.004 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:47:50.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:50.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:50.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:47:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:50.723 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:51.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:50 smithi096 ceph-mon[29288]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:51.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:50 smithi096 ceph-mon[29288]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c;count:3 2024-04-25T20:47:51.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:50 smithi096 ceph-mon[29288]: Deploying daemon mon.c on smithi139 2024-04-25T20:47:52.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:51 smithi096 ceph-mon[29288]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 -- 172.21.15.139:0/2937279329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 msgr2=0x7f412c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 --2- 172.21.15.139:0/2937279329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c101990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f411c009910 tx=0x7f411c009c20 comp rx=0 tx=0).stop 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 -- 172.21.15.139:0/2937279329 shutdown_connections 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 --2- 172.21.15.139:0/2937279329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c101990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 -- 172.21.15.139:0/2937279329 >> 172.21.15.139:0/2937279329 conn(0x7f412c0fce60 msgr2=0x7f412c0ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 -- 172.21.15.139:0/2937279329 shutdown_connections 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 -- 172.21.15.139:0/2937279329 wait complete. 2024-04-25T20:47:53.670 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.668+0000 7f413121e700 1 Processor -- start 2024-04-25T20:47:53.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.669+0000 7f413121e700 1 -- start start 2024-04-25T20:47:53.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.669+0000 7f413121e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:53.671 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.669+0000 7f413121e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f412c10b830 con 0x7f412c1015b0 2024-04-25T20:47:53.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.669+0000 7f412ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:53.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.669+0000 7f412ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.139:42156/0 (socket says 172.21.15.139:42156) 2024-04-25T20:47:53.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.669+0000 7f412ad9d700 1 -- 172.21.15.139:0/402398708 learned_addr learned my addr 172.21.15.139:0/402398708 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:47:53.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f412ad9d700 1 -- 172.21.15.139:0/402398708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f412c198400 con 0x7f412c1015b0 2024-04-25T20:47:53.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f412ad9d700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c197ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f411c0056d0 tx=0x7f411c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:53.672 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c026070 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f412c1986f0 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f411c00bba0 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f411c003920 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.670+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f412c19c740 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.671+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f411c029070 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.671+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f412c102a90 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.671+0000 7f4113fff700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4114039650 0x7f411403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.671+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f411c01e070 con 0x7f412c1015b0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.671+0000 7f412a59c700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4114039650 0x7f411403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:47:53.673 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.672+0000 7f412a59c700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4114039650 0x7f411403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4120006fd0 tx=0x7f4120006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:47:53.677 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.677+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f411c013070 con 0x7f412c1015b0 2024-04-25T20:47:53.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.979+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f412c1991d0 con 0x7f412c1015b0 2024-04-25T20:47:53.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.980+0000 7f4113fff700 1 -- 172.21.15.139:0/402398708 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f411c030020 con 0x7f412c1015b0 2024-04-25T20:47:53.981 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:47:53.981 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":1,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","modified":"2024-04-25T20:46:10.879735Z","created":"2024-04-25T20:46:10.879735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4114039650 msgr2=0x7f411403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4114039650 0x7f411403bb10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f4120006fd0 tx=0x7f4120006e40 comp rx=0 tx=0).stop 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 msgr2=0x7f412c197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c197ec0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f411c0056d0 tx=0x7f411c0057b0 comp rx=0 tx=0).stop 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 shutdown_connections 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4114039650 0x7f411403bb10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:53.984 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 --2- 172.21.15.139:0/402398708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f412c1015b0 0x7f412c197ec0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:47:53.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.982+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 >> 172.21.15.139:0/402398708 conn(0x7f412c0fce60 msgr2=0x7f412c06a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:47:53.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.983+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 shutdown_connections 2024-04-25T20:47:53.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:47:53.983+0000 7f413121e700 1 -- 172.21.15.139:0/402398708 wait complete. 2024-04-25T20:47:53.985 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 1 2024-04-25T20:47:54.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:54 smithi096 ceph-mon[29288]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:54.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:47:54 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/402398708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:47:54.827 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:54 smithi139 podman[33008]: 2024-04-25 20:47:54.71899078 +0000 UTC m=+0.018222101 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:47:54.827 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:54 smithi139 podman[33008]: 2024-04-25 20:47:54.827122592 +0000 UTC m=+0.126353960 container create 7941e855b680ac30167dad977b9dfa4796b05b82644206b9b661abba9784f082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T20:47:55.078 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 podman[33008]: 2024-04-25 20:47:55.046039849 +0000 UTC m=+0.345271171 container init 7941e855b680ac30167dad977b9dfa4796b05b82644206b9b661abba9784f082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T20:47:55.078 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 podman[33008]: 2024-04-25 20:47:55.049145699 +0000 UTC m=+0.348377022 container start 7941e855b680ac30167dad977b9dfa4796b05b82644206b9b661abba9784f082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:47:55.078 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: set uid:gid to 167:167 (ceph:ceph) 2024-04-25T20:47:55.078 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: ceph version 19.0.0-3156-gd69f13d8 (d69f13d827952bb07bb9b7973bca14f97d764d6f) squid (dev), process ceph-mon, pid 2 2024-04-25T20:47:55.078 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pidfile_write: ignore empty --pid-file 2024-04-25T20:47:55.078 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: load: jerasure load: lrc 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: RocksDB version: 7.9.2 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Git sha 0 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Compile date 2024-04-22 10:57:25 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: DB SUMMARY 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: DB Session ID: PDSQ4AHA7E6OJ5339WIO 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: CURRENT file: CURRENT 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: IDENTITY file: IDENTITY 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-25T20:47:55.079 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.error_if_exists: 0 2024-04-25T20:47:55.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.create_if_missing: 0 2024-04-25T20:47:55.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.paranoid_checks: 1 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.env: 0x55f4a8164cc0 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.fs: PosixFileSystem 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.info_log: 0x55f4a8d97be0 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_file_opening_threads: 16 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.statistics: (nil) 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.use_fsync: 0 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_log_file_size: 0 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-25T20:47:55.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.keep_log_file_num: 1000 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.recycle_log_file_num: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.allow_fallocate: 1 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.allow_mmap_reads: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.allow_mmap_writes: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.use_direct_reads: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.create_missing_column_families: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.db_log_dir: 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.wal_dir: 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-25T20:47:55.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.advise_random_on_open: 1 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.db_write_buffer_size: 0 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.write_buffer_manager: 0x55f4a8d2b0e0 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.rate_limiter: (nil) 2024-04-25T20:47:55.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.wal_recovery_mode: 2 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.enable_thread_tracking: 0 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.enable_pipelined_write: 0 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.unordered_write: 0 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.row_cache: None 2024-04-25T20:47:55.378 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.wal_filter: None 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.allow_ingest_behind: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.two_write_queues: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.manual_wal_flush: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.wal_compression: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.atomic_flush: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.log_readahead_size: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.best_efforts_recovery: 0 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-25T20:47:55.379 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.allow_data_in_errors: 0 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.db_host_id: __hostname__ 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_background_jobs: 2 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_background_compactions: -1 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_subcompactions: 1 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_total_wal_size: 0 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-25T20:47:55.380 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_open_files: -1 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bytes_per_sync: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_readahead_size: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_background_flushes: -1 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Compression algorithms supported: 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kZSTD supported: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kXpressCompression supported: 0 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kLZ4HCCompression supported: 1 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kZlibCompression supported: 1 2024-04-25T20:47:55.381 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kSnappyCompression supported: 1 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kLZ4Compression supported: 1 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: kBZip2Compression supported: 0 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.merge_operator: 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_filter: None 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_filter_factory: None 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.sst_partitioner_factory: None 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-25T20:47:55.382 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f4a8d97dc0) 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: cache_index_and_filter_blocks: 1 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: pin_top_level_index_and_filter: 1 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: index_type: 0 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: data_block_index_type: 0 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: index_shortening: 1 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: checksum: 4 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: no_block_cache: 0 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_cache: 0x55f4a8d5d1f0 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_cache_name: BinnedLRUCache 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_cache_options: 2024-04-25T20:47:55.383 INFO:journalctl@ceph.mon.c.smithi139.stdout: capacity : 536870912 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: num_shard_bits : 4 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: strict_capacity_limit : 0 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: high_pri_pool_ratio: 0.000 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_cache_compressed: (nil) 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: persistent_cache: (nil) 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_size: 4096 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_size_deviation: 10 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_restart_interval: 16 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: index_block_restart_interval: 1 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: metadata_block_size: 4096 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: partition_filters: 0 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: use_delta_encoding: 1 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: filter_policy: bloomfilter 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: whole_key_filtering: 1 2024-04-25T20:47:55.384 INFO:journalctl@ceph.mon.c.smithi139.stdout: verify_compression: 0 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: read_amp_bytes_per_bit: 0 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: format_version: 5 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: enable_index_compression: 1 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: block_align: 0 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: max_auto_readahead_size: 262144 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: prepopulate_block_cache: 0 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: initial_auto_readahead_size: 8192 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout: num_file_reads_for_auto_readahead: 2 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.write_buffer_size: 33554432 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_write_buffer_number: 2 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression: NoCompression 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression: Disabled 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.prefix_extractor: nullptr 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-25T20:47:55.385 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.num_levels: 7 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-25T20:47:55.386 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.level: 32767 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.strategy: 0 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.enabled: false 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.target_file_size_base: 67108864 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-25T20:47:55.387 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-25T20:47:55.388 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.arena_block_size: 1048576 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.disable_auto_compactions: 0 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-25T20:47:55.389 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.inplace_update_support: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.bloom_locality: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.max_successive_merges: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.paranoid_file_checks: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.force_consistency_checks: 1 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.report_bg_io_stats: 0 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.ttl: 2592000 2024-04-25T20:47:55.390 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.enable_blob_files: false 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.min_blob_size: 0 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.blob_file_size: 268435456 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.blob_file_starting_level: 0 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-25T20:47:55.391 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b45f5c07-c790-4379-a785-004cd5a8f979 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078075079087, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078075079504, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714078075, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b45f5c07-c790-4379-a785-004cd5a8f979", "db_session_id": "PDSQ4AHA7E6OJ5339WIO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714078075079584, "job": 1, "event": "recovery_finished"} 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f4a8e72000 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: DB pointer 0x55f4a8e5c000 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout: ** DB Stats ** 2024-04-25T20:47:55.392 INFO:journalctl@ceph.mon.c.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: ** Compaction Stats [default] ** 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:47:55.393 INFO:journalctl@ceph.mon.c.smithi139.stdout: 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: ** Compaction Stats [default] ** 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-25T20:47:55.394 INFO:journalctl@ceph.mon.c.smithi139.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: Cumulative compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: Interval compaction: 0.00 GB write, 0.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: Block cache BinnedLRUCache@0x55f4a8d5d1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: using public_addr v2:172.21.15.139:0/0 -> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: starting mon.c rank -1 at public addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] at bind addrs [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(???) e0 preinit fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).mds e1 new map 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: e1 2024-04-25T20:47:55.395 INFO:journalctl@ceph.mon.c.smithi139.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout: legacy client fscid: -1 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout: 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout: No filesystems configured 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:47:55.396 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi139", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:55.397 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Deploying cephadm binary to smithi139 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Added host smithi139 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.398 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1453273568' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1453273568' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi139", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:55.399 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:55.400 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Saving service mon spec with placement smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c;count:3 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: Deploying daemon mon.c on smithi139 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/402398708' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 ceph-mon[33027]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 bash[33008]: 7941e855b680ac30167dad977b9dfa4796b05b82644206b9b661abba9784f082 2024-04-25T20:47:55.401 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:47:55 smithi139 systemd[1]: Started Ceph mon.c for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:47:55.604 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:47:55.604 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:47:55.837 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:48:00.324 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 -- 172.21.15.139:0/1253357098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 msgr2=0x7f132c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 --2- 172.21.15.139:0/1253357098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f132c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1344078b70 tx=0x7f133400feb0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 -- 172.21.15.139:0/1253357098 shutdown_connections 2024-04-25T20:48:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 --2- 172.21.15.139:0/1253357098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f132c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 -- 172.21.15.139:0/1253357098 >> 172.21.15.139:0/1253357098 conn(0x7f13440778c0 msgr2=0x7f1344077cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 -- 172.21.15.139:0/1253357098 shutdown_connections 2024-04-25T20:48:00.325 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.323+0000 7f134b5c3700 1 -- 172.21.15.139:0/1253357098 wait complete. 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f134b5c3700 1 Processor -- start 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f134b5c3700 1 -- start start 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f134b5c3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f1344197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f134b5c3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f134b5c3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f134410a480 con 0x7f132c0035c0 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f134b5c3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f134410a300 con 0x7f1344198450 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f1348b5e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.324+0000 7f1348b5e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50710/0 (socket says 172.21.15.139:50710) 2024-04-25T20:48:00.327 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f1348b5e700 1 -- 172.21.15.139:0/223588524 learned_addr learned my addr 172.21.15.139:0/223588524 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f134935f700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f1344197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f1348b5e700 1 -- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 msgr2=0x7f134419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f1348b5e700 1 -- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 msgr2=0x7f134419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f1348b5e700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f1348b5e700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f134935f700 1 -- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 msgr2=0x7f134419c7e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f134935f700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f134935f700 1 -- 172.21.15.139:0/223588524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1344198a90 con 0x7f132c0035c0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.325+0000 7f134935f700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f1344197f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1334010010 tx=0x7f13340116a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.326+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1334013960 con 0x7f132c0035c0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.326+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13340044e0 con 0x7f132c0035c0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.326+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f134419cd20 con 0x7f132c0035c0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.326+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f1334011b80 con 0x7f132c0035c0 2024-04-25T20:48:00.328 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.326+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f134419d300 con 0x7f132c0035c0 2024-04-25T20:48:00.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.327+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1334023070 con 0x7f132c0035c0 2024-04-25T20:48:00.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.327+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f134406a110 con 0x7f132c0035c0 2024-04-25T20:48:00.329 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.327+0000 7f13427fc700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1330039440 0x7f133003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:00.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.327+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1334020050 con 0x7f132c0035c0 2024-04-25T20:48:00.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.327+0000 7f1348b5e700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1330039440 0x7f133003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:00.330 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.328+0000 7f1348b5e700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1330039440 0x7f133003b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1338009400 tx=0x7f1338007040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:00.333 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.333+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1334028020 con 0x7f132c0035c0 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: mon.a calling monitor election 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: mon.c calling monitor election 2024-04-25T20:48:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: monmap epoch 2 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: last_changed 2024-04-25T20:47:55.102493+0000 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: min_mon_release 19 (squid) 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: election_strategy: 1 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:00.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: fsmap 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: mgrmap e13: x(active, since 50s) 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: overall HEALTH_OK 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:00.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: mon.a calling monitor election 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: mon.c calling monitor election 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: monmap epoch 2 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: last_changed 2024-04-25T20:47:55.102493+0000 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: min_mon_release 19 (squid) 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: election_strategy: 1 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: fsmap 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:00.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: mgrmap e13: x(active, since 50s) 2024-04-25T20:48:00.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: overall HEALTH_OK 2024-04-25T20:48:00.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:00.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:00.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:00.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:00.502 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:00.615 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.614+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f134419d5b0 con 0x7f132c0035c0 2024-04-25T20:48:00.616 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.615+0000 7f13427fc700 1 -- 172.21.15.139:0/223588524 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f1334024e20 con 0x7f132c0035c0 2024-04-25T20:48:00.616 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:48:00.616 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":2,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","modified":"2024-04-25T20:47:55.102493Z","created":"2024-04-25T20:46:10.879735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T20:48:00.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1330039440 msgr2=0x7f133003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:00.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1330039440 0x7f133003b900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1338009400 tx=0x7f1338007040 comp rx=0 tx=0).stop 2024-04-25T20:48:00.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 msgr2=0x7f1344197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:00.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f1344197f10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1334010010 tx=0x7f13340116a0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.618 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 shutdown_connections 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1330039440 0x7f133003b900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f132c0035c0 0x7f1344197f10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 --2- 172.21.15.139:0/223588524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1344198450 0x7f134419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 >> 172.21.15.139:0/223588524 conn(0x7f13440778c0 msgr2=0x7f13440fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 shutdown_connections 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:00.617+0000 7f134b5c3700 1 -- 172.21.15.139:0/223588524 wait complete. 2024-04-25T20:48:00.619 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 2 2024-04-25T20:48:01.188 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:01 smithi139 ceph-mon[33027]: Deploying daemon mon.b on smithi097 2024-04-25T20:48:01.188 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:01 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/223588524' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:48:01.188 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:01.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:01 smithi096 ceph-mon[29288]: Deploying daemon mon.b on smithi097 2024-04-25T20:48:01.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:01 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/223588524' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:48:01.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:02.191 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-25T20:48:02.191 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-25T20:48:02.435 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:48:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:02 smithi096 ceph-mon[29288]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:02.570 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:02 smithi139 ceph-mon[33027]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.975+0000 7f07141ad700 1 -- 172.21.15.139:0/582667983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c108630 msgr2=0x7f070c108a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.975+0000 7f07141ad700 1 --2- 172.21.15.139:0/582667983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c108630 0x7f070c108a10 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f0708009910 tx=0x7f0708009c20 comp rx=0 tx=0).stop 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 -- 172.21.15.139:0/582667983 shutdown_connections 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 --2- 172.21.15.139:0/582667983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c108630 0x7f070c108a10 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 -- 172.21.15.139:0/582667983 >> 172.21.15.139:0/582667983 conn(0x7f070c0fe050 msgr2=0x7f070c100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 -- 172.21.15.139:0/582667983 shutdown_connections 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 -- 172.21.15.139:0/582667983 wait complete. 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 Processor -- start 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.976+0000 7f07141ad700 1 -- start start 2024-04-25T20:48:03.980 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f07141ad700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f07141ad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c075280 0x7f070c075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f07141ad700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f070c10caa0 con 0x7f070c075280 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f07141ad700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f070c10c920 con 0x7f070c108630 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:50726/0 (socket says 172.21.15.139:50726) 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 -- 172.21.15.139:0/1108605146 learned_addr learned my addr 172.21.15.139:0/1108605146 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711748700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c075280 0x7f070c075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 -- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 msgr2=0x7f070c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 -- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 msgr2=0x7f070c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711f49700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711748700 1 -- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 msgr2=0x7f070c074d40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711748700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:03.981 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711748700 1 -- 172.21.15.139:0/1108605146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f070c076c60 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.977+0000 7f0711748700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c075280 0x7f070c075700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00b4d0 tx=0x7f06fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.978+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f06fc011630 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.978+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06fc011c80 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.978+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f070c1a4410 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.978+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f06fc010da0 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.978+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f070c1a4950 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.979+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f06fc018020 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.979+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f070c103ca0 con 0x7f070c075280 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.979+0000 7f0702ffd700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f06f80393f0 0x7f06f803b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:03.982 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.979+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f06fc04c230 con 0x7f070c075280 2024-04-25T20:48:03.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.979+0000 7f0711f49700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f06f80393f0 0x7f06f803b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:03.983 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.980+0000 7f0711f49700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f06f80393f0 0x7f06f803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0708009750 tx=0x7f070801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:03.985 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:03.984+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06fc01f020 con 0x7f070c075280 2024-04-25T20:48:04.213 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:04.212+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06fc0177d0 con 0x7f070c075280 2024-04-25T20:48:04.269 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:04.267+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f070c1a4d00 con 0x7f070c075280 2024-04-25T20:48:04.350 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:04 smithi097 systemd[1]: Started Ceph mon.b for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:48:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:04 smithi096 ceph-mon[29288]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:04 smithi139 ceph-mon[33027]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:09.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.221+0000 7f0702ffd700 1 -- 172.21.15.139:0/1108605146 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f06fc03e510 con 0x7f070c075280 2024-04-25T20:48:09.222 INFO:teuthology.orchestra.run.smithi139.stdout: 2024-04-25T20:48:09.223 INFO:teuthology.orchestra.run.smithi139.stdout:{"epoch":3,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","modified":"2024-04-25T20:48:04.211890Z","created":"2024-04-25T20:46:10.879735Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:3300","nonce":0},{"type":"v1","addr":"172.21.15.139:6789","nonce":0}]},"addr":"172.21.15.139:6789/0","public_addr":"172.21.15.139:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f06f80393f0 msgr2=0x7f06f803b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f06f80393f0 0x7f06f803b8b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0708009750 tx=0x7f070801a040 comp rx=0 tx=0).stop 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c075280 msgr2=0x7f070c075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c075280 0x7f070c075700 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f06fc00b4d0 tx=0x7f06fc00b890 comp rx=0 tx=0).stop 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 shutdown_connections 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f06f80393f0 0x7f06f803b8b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f070c108630 0x7f070c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 --2- 172.21.15.139:0/1108605146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f070c075280 0x7f070c075700 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.224+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 >> 172.21.15.139:0/1108605146 conn(0x7f070c0fe050 msgr2=0x7f070c0fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.225+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 shutdown_connections 2024-04-25T20:48:09.226 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:09.225+0000 7f07141ad700 1 -- 172.21.15.139:0/1108605146 wait complete. 2024-04-25T20:48:09.227 INFO:teuthology.orchestra.run.smithi139.stderr:dumped monmap epoch 3 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: mon.c calling monitor election 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: mon.a calling monitor election 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/1108605146' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.532 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: monmap epoch 3 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: last_changed 2024-04-25T20:48:04.211890+0000 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: min_mon_release 19 (squid) 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: election_strategy: 1 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:09.533 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T20:48:09.534 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: fsmap 2024-04-25T20:48:09.534 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:09.534 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: mgrmap e13: x(active, since 60s) 2024-04-25T20:48:09.534 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: overall HEALTH_OK 2024-04-25T20:48:09.534 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:09.534 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: mon.c calling monitor election 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: mon.a calling monitor election 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/1108605146' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: monmap epoch 3 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: last_changed 2024-04-25T20:48:04.211890+0000 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: min_mon_release 19 (squid) 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: election_strategy: 1 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:09.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: fsmap 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: mgrmap e13: x(active, since 60s) 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: overall HEALTH_OK 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:09.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:09.819 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-25T20:48:09.820 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-25T20:48:10.066 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:48:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:10 smithi096 ceph-mon[29288]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:10.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:10 smithi139 ceph-mon[33027]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.c calling monitor election 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.a calling monitor election 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/1108605146' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: monmap epoch 3 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: last_changed 2024-04-25T20:48:04.211890+0000 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: min_mon_release 19 (squid) 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: election_strategy: 1 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: fsmap 2024-04-25T20:48:11.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mgrmap e13: x(active, since 60s) 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:11.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:11.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 -- 172.21.15.96:0/830386251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 msgr2=0x7fc00c101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:11.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 --2- 172.21.15.96:0/830386251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c101990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009910 tx=0x7fbffc009c20 comp rx=0 tx=0).stop 2024-04-25T20:48:11.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 -- 172.21.15.96:0/830386251 shutdown_connections 2024-04-25T20:48:11.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 --2- 172.21.15.96:0/830386251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c101990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 -- 172.21.15.96:0/830386251 >> 172.21.15.96:0/830386251 conn(0x7fc00c0fce40 msgr2=0x7fc00c0ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:11.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 -- 172.21.15.96:0/830386251 shutdown_connections 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.765+0000 7fc0119cc700 1 -- 172.21.15.96:0/830386251 wait complete. 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 Processor -- start 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 -- start start 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c1975e0 0x7fc00c19b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c19beb0 0x7fc00c19c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10b830 con 0x7fc00c1015b0 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10b6b0 con 0x7fc00c19beb0 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc0119cc700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc00c10b9b0 con 0x7fc00c1975e0 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc00affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc00affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40352/0 (socket says 172.21.15.96:40352) 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc00affd700 1 -- 172.21.15.96:0/1294171943 learned_addr learned my addr 172.21.15.96:0/1294171943 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.766+0000 7fc00a7fc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c1975e0 0x7fc00c19b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:11.768 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00b7fe700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c19beb0 0x7fc00c19c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00affd700 1 -- 172.21.15.96:0/1294171943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c1975e0 msgr2=0x7fc00c19b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00affd700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c1975e0 0x7fc00c19b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00affd700 1 -- 172.21.15.96:0/1294171943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c19beb0 msgr2=0x7fc00c19c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00affd700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c19beb0 0x7fc00c19c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00affd700 1 -- 172.21.15.96:0/1294171943 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc00c1a3c20 con 0x7fc00c1015b0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc00affd700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c1970a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbffc004f80 tx=0x7fbffc005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc026070 con 0x7fc00c1015b0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbffc00b6f0 con 0x7fc00c1015b0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc00c1a3e50 con 0x7fc00c1015b0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbffc003970 con 0x7fc00c1015b0 2024-04-25T20:48:11.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.767+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc00c1a42a0 con 0x7fc00c1015b0 2024-04-25T20:48:11.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.768+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fbffc029070 con 0x7fc00c1015b0 2024-04-25T20:48:11.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.768+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc00c0640a0 con 0x7fc00c1015b0 2024-04-25T20:48:11.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.768+0000 7fc0109ca700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbff4039ae0 0x7fbff403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:11.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.768+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbffc04d4d0 con 0x7fc00c1015b0 2024-04-25T20:48:11.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.768+0000 7fc00a7fc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbff4039ae0 0x7fbff403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:11.770 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.769+0000 7fc00a7fc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbff4039ae0 0x7fbff403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc000009850 tx=0x7fc000008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:11.773 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.774+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbffc013070 con 0x7fc00c1015b0 2024-04-25T20:48:11.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.985+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fc00c1a46b0 con 0x7fc00c1015b0 2024-04-25T20:48:11.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.985+0000 7fc0109ca700 1 -- 172.21.15.96:0/1294171943 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fbffc02a570 con 0x7fc00c1015b0 2024-04-25T20:48:11.986 INFO:teuthology.orchestra.run.smithi096.stdout:# minimal ceph.conf for b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:11.986 INFO:teuthology.orchestra.run.smithi096.stdout:[global] 2024-04-25T20:48:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: fsid = b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:11.986 INFO:teuthology.orchestra.run.smithi096.stdout: mon_host = [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] 2024-04-25T20:48:11.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: mon.b calling monitor election 2024-04-25T20:48:11.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:48:11.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:48:11.986 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: mon.b calling monitor election 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: mon.c calling monitor election 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: mon.a calling monitor election 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: monmap epoch 3 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: last_changed 2024-04-25T20:48:04.211890+0000 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: min_mon_release 19 (squid) 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: election_strategy: 1 2024-04-25T20:48:11.987 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: fsmap 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: mgrmap e13: x(active, since 62s) 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: overall HEALTH_OK 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.988 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:11.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:11.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:11.989 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.987+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbff4039ae0 msgr2=0x7fbff403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.987+0000 7fc0119cc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbff4039ae0 0x7fbff403bfa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc000009850 tx=0x7fc000008040 comp rx=0 tx=0).stop 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.987+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 msgr2=0x7fc00c1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.987+0000 7fc0119cc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c1970a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbffc004f80 tx=0x7fbffc005060 comp rx=0 tx=0).stop 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 shutdown_connections 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbff4039ae0 0x7fbff403bfa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc00c19beb0 0x7fc00c19c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc00c1015b0 0x7fc00c1970a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 --2- 172.21.15.96:0/1294171943 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc00c1975e0 0x7fc00c19b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 >> 172.21.15.96:0/1294171943 conn(0x7fc00c0fce40 msgr2=0x7fc00c102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 shutdown_connections 2024-04-25T20:48:11.990 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:11.988+0000 7fc0119cc700 1 -- 172.21.15.96:0/1294171943 wait complete. 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.b calling monitor election 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.b calling monitor election 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.c calling monitor election 2024-04-25T20:48:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.a calling monitor election 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: monmap epoch 3 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: last_changed 2024-04-25T20:48:04.211890+0000 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: min_mon_release 19 (squid) 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: election_strategy: 1 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: fsmap 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: mgrmap e13: x(active, since 62s) 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T20:48:12.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:12.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: mon.b calling monitor election 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: mon.b calling monitor election 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: mon.c calling monitor election 2024-04-25T20:48:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: mon.a calling monitor election 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: monmap epoch 3 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: last_changed 2024-04-25T20:48:04.211890+0000 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: created 2024-04-25T20:46:10.879735+0000 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: min_mon_release 19 (squid) 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: election_strategy: 1 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: 0: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.a 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: 1: [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] mon.c 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: 2: [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon.b 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: fsmap 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: osdmap e4: 0 total, 0 up, 0 in 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: mgrmap e13: x(active, since 62s) 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: overall HEALTH_OK 2024-04-25T20:48:12.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:13.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:12 smithi097 ceph-mon[32667]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:48:13.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:12 smithi097 ceph-mon[32667]: Reconfiguring daemon mon.a on smithi096 2024-04-25T20:48:13.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:12 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1294171943' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:13.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:13.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:12 smithi139 ceph-mon[33027]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:48:13.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:12 smithi139 ceph-mon[33027]: Reconfiguring daemon mon.a on smithi096 2024-04-25T20:48:13.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:12 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1294171943' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:13.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:13.210 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:12 smithi096 ceph-mon[29288]: Reconfiguring mon.a (unknown last config time)... 2024-04-25T20:48:13.210 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:12 smithi096 ceph-mon[29288]: Reconfiguring daemon mon.a on smithi096 2024-04-25T20:48:13.211 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:12 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1294171943' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:13.211 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:48:13.238 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-25T20:48:13.238 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:48:13.238 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:48:13.266 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:48:13.266 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:48:13.330 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:48:13.330 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:48:13.362 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:48:13.362 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:48:13.430 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:48:13.430 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.conf 2024-04-25T20:48:13.461 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:48:13.461 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:48:13.528 INFO:tasks.cephadm:Adding mgr.x on smithi096 2024-04-25T20:48:13.528 INFO:tasks.cephadm:Adding mgr.y on smithi097 2024-04-25T20:48:13.529 INFO:tasks.cephadm:Adding mgr.z on smithi139 2024-04-25T20:48:13.529 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi096=x;smithi097=y;smithi139=z' 2024-04-25T20:48:13.805 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:48:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:13 smithi096 ceph-mon[29288]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:14.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:14.034 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:13 smithi139 ceph-mon[33027]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:14.034 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:14.034 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:14.034 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:14.034 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:14.035 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:13 smithi097 ceph-mon[32667]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:15.314 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 -- 172.21.15.139:0/1078793628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 msgr2=0x7f9940101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 --2- 172.21.15.139:0/1078793628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940101900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f993000b0a0 tx=0x7f993000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 -- 172.21.15.139:0/1078793628 shutdown_connections 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 --2- 172.21.15.139:0/1078793628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940101900 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 -- 172.21.15.139:0/1078793628 >> 172.21.15.139:0/1078793628 conn(0x7f99400fca80 msgr2=0x7f99400feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 -- 172.21.15.139:0/1078793628 shutdown_connections 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.313+0000 7f994603b700 1 -- 172.21.15.139:0/1078793628 wait complete. 2024-04-25T20:48:15.315 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 Processor -- start 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 -- start start 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9940198320 0x7f994019d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 0x7f994019f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99401073d0 con 0x7f99401014e0 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9940107250 con 0x7f9940198320 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f994603b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9940107550 con 0x7f99401987a0 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f993ffff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 0x7f994019f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:15.317 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f993ffff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 0x7f994019f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:53016/0 (socket says 172.21.15.139:53016) 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f993ffff700 1 -- 172.21.15.139:0/2911296716 learned_addr learned my addr 172.21.15.139:0/2911296716 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f993f7fe700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.314+0000 7f993effd700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9940198320 0x7f994019d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993ffff700 1 -- 172.21.15.139:0/2911296716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9940198320 msgr2=0x7f994019d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993ffff700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9940198320 0x7f994019d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993ffff700 1 -- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 msgr2=0x7f9940197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993ffff700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993ffff700 1 -- 172.21.15.139:0/2911296716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99401a01f0 con 0x7f99401987a0 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993f7fe700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993ffff700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 0x7f994019f920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f993800b550 tx=0x7f993800b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938015020 con 0x7f99401987a0 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99401a04e0 con 0x7f99401987a0 2024-04-25T20:48:15.318 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9938011ce0 con 0x7f99401987a0 2024-04-25T20:48:15.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.315+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99401a0af0 con 0x7f99401987a0 2024-04-25T20:48:15.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.316+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9938010cb0 con 0x7f99401987a0 2024-04-25T20:48:15.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.316+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9938018020 con 0x7f99401987a0 2024-04-25T20:48:15.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.316+0000 7f993cff9700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f992c039ae0 0x7f992c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:15.319 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.316+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99400640a0 con 0x7f99401987a0 2024-04-25T20:48:15.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.316+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f9938011830 con 0x7f99401987a0 2024-04-25T20:48:15.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.317+0000 7f993f7fe700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f992c039ae0 0x7f992c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:15.320 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.317+0000 7f993f7fe700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f992c039ae0 0x7f992c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9930015040 tx=0x7f9930006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:15.323 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.322+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f993801f020 con 0x7f99401987a0 2024-04-25T20:48:15.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:15 smithi139 ceph-mon[33027]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:48:15.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:15 smithi139 ceph-mon[33027]: Reconfiguring daemon mon.b on smithi097 2024-04-25T20:48:15.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:15 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:15.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:15 smithi096 ceph-mon[29288]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:48:15.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:15 smithi096 ceph-mon[29288]: Reconfiguring daemon mon.b on smithi097 2024-04-25T20:48:15.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:15 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:15.537 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.536+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi139=z", "target": ["mon-mgr", ""]}) -- 0x7f9940062e80 con 0x7f992c039ae0 2024-04-25T20:48:15.543 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.542+0000 7f993cff9700 1 -- 172.21.15.139:0/2911296716 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f9940062e80 con 0x7f992c039ae0 2024-04-25T20:48:15.543 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mgr update... 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f992c039ae0 msgr2=0x7f992c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f992c039ae0 0x7f992c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9930015040 tx=0x7f9930006210 comp rx=0 tx=0).stop 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 msgr2=0x7f994019f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 0x7f994019f920 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f993800b550 tx=0x7f993800b860 comp rx=0 tx=0).stop 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 shutdown_connections 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f992c039ae0 0x7f992c03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f99401987a0 0x7f994019f920 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.544+0000 7f994603b700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401014e0 0x7f9940197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.545+0000 7f994603b700 1 --2- 172.21.15.139:0/2911296716 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9940198320 0x7f994019d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.545+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 >> 172.21.15.139:0/2911296716 conn(0x7f99400fca80 msgr2=0x7f994018ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:15.546 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.545+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 shutdown_connections 2024-04-25T20:48:15.547 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:48:15.545+0000 7f994603b700 1 -- 172.21.15.139:0/2911296716 wait complete. 2024-04-25T20:48:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:15 smithi097 ceph-mon[32667]: Reconfiguring mon.b (monmap changed)... 2024-04-25T20:48:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:15 smithi097 ceph-mon[32667]: Reconfiguring daemon mon.b on smithi097 2024-04-25T20:48:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:15 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.174 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.y.service 2024-04-25T20:48:16.177 DEBUG:teuthology.orchestra.run.smithi139:mgr.z> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.z.service 2024-04-25T20:48:16.179 INFO:tasks.cephadm:Deploying OSDs... 2024-04-25T20:48:16.179 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:48:16.180 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:48:16.198 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:48:16.198 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 20:47:21.830130978 +0000 2024-04-25T20:48:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 20:42:56.711965358 +0000 2024-04-25T20:48:16.258 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 20:42:56.711965358 +0000 2024-04-25T20:48:16.258 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 20:42:56.711965358 +0000 2024-04-25T20:48:16.258 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:48:16.324 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T20:48:16.325 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T20:48:16.325 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000236228 s, 2.2 MB/s 2024-04-25T20:48:16.326 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:48:16.383 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-04-25T20:48:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:48:16.440 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:16.440 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T20:48:16.440 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:16.440 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:16.441 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 20:47:21.858130359 +0000 2024-04-25T20:48:16.441 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 20:42:57.056957319 +0000 2024-04-25T20:48:16.441 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 20:42:57.056957319 +0000 2024-04-25T20:48:16.441 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 20:42:57.056957319 +0000 2024-04-25T20:48:16.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:48:16.450 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:16.450 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.450 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.450 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:48:16.451 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:16.451 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:16.451 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:16.451 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: Reconfiguring daemon mon.c on smithi139 2024-04-25T20:48:16.451 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.505 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T20:48:16.505 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T20:48:16.505 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000205811 s, 2.5 MB/s 2024-04-25T20:48:16.506 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:48:16.564 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-04-25T20:48:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: Reconfiguring daemon mon.c on smithi139 2024-04-25T20:48:16.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 20:47:21.895129541 +0000 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 20:42:57.400949305 +0000 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 20:42:57.400949305 +0000 2024-04-25T20:48:16.620 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 20:42:57.400949305 +0000 2024-04-25T20:48:16.620 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:48:16.680 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:16.680 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: Reconfiguring mon.c (monmap changed)... 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: Reconfiguring daemon mon.c on smithi139 2024-04-25T20:48:16.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:16.683 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T20:48:16.683 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T20:48:16.683 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000177892 s, 2.9 MB/s 2024-04-25T20:48:16.684 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:48:16.741 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 20:47:21.928128812 +0000 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 20:42:57.754941057 +0000 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 20:42:57.754941057 +0000 2024-04-25T20:48:16.797 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 20:42:57.754941057 +0000 2024-04-25T20:48:16.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:48:16.859 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-04-25T20:48:16.859 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-04-25T20:48:16.859 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000181136 s, 2.8 MB/s 2024-04-25T20:48:16.861 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:48:16.917 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2024-04-25T20:48:16.917 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:48:16.936 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:48:16.936 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2024-04-25T20:48:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:48:16.993 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:16.993 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T20:48:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 20:48:03.681098634 +0000 2024-04-25T20:48:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 20:43:14.396084432 +0000 2024-04-25T20:48:16.994 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 20:43:14.396084432 +0000 2024-04-25T20:48:16.994 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 20:43:14.396084432 +0000 2024-04-25T20:48:16.994 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:48:17.060 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T20:48:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T20:48:17.061 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000284977 s, 1.8 MB/s 2024-04-25T20:48:17.062 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:48:17.120 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 20:48:03.682098616 +0000 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 20:43:14.732078188 +0000 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 20:43:14.732078188 +0000 2024-04-25T20:48:17.177 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 20:43:14.732078188 +0000 2024-04-25T20:48:17.177 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:48:17.241 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T20:48:17.242 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T20:48:17.242 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.00020457 s, 2.5 MB/s 2024-04-25T20:48:17.243 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:48:17.300 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 20:48:03.682098616 +0000 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 20:43:15.071071888 +0000 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 20:43:15.071071888 +0000 2024-04-25T20:48:17.356 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 20:43:15.071071888 +0000 2024-04-25T20:48:17.357 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:48:17.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:17 smithi097 ceph-mon[32667]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi139=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:17.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:17 smithi097 ceph-mon[32667]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi139=z;count:3 2024-04-25T20:48:17.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:17 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:17.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:17 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T20:48:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T20:48:17.386 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000172059 s, 3.0 MB/s 2024-04-25T20:48:17.387 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:48:17.443 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2024-04-25T20:48:17.497 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2024-04-25 20:48:03.682098616 +0000 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2024-04-25 20:43:15.423065346 +0000 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2024-04-25 20:43:15.423065346 +0000 2024-04-25T20:48:17.498 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: 2024-04-25 20:43:15.423065346 +0000 2024-04-25T20:48:17.498 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:48:17.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2024-04-25T20:48:17.560 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2024-04-25T20:48:17.560 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000220173 s, 2.3 MB/s 2024-04-25T20:48:17.561 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:48:17.617 DEBUG:teuthology.orchestra.run.smithi139:> set -ex 2024-04-25T20:48:17.617 DEBUG:teuthology.orchestra.run.smithi139:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T20:48:17.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:17 smithi139 ceph-mon[33027]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi139=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:17.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:17 smithi139 ceph-mon[33027]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi139=z;count:3 2024-04-25T20:48:17.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:17 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:17 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:17.635 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T20:48:17.635 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_1 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 20:47:53.999823883 +0000 2024-04-25T20:48:17.691 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 20:43:16.327968322 +0000 2024-04-25T20:48:17.692 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 20:43:16.327968322 +0000 2024-04-25T20:48:17.692 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 20:43:16.327968322 +0000 2024-04-25T20:48:17.692 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T20:48:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:17 smithi096 ceph-mon[29288]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi096=x;smithi097=y;smithi139=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:17 smithi096 ceph-mon[29288]: Saving service mgr spec with placement smithi096=x;smithi097=y;smithi139=z;count:3 2024-04-25T20:48:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:17 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:17 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:17.757 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T20:48:17.757 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T20:48:17.757 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000275495 s, 1.9 MB/s 2024-04-25T20:48:17.759 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T20:48:17.816 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_2 2024-04-25T20:48:17.870 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 20:47:54.000823862 +0000 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 20:43:16.671960502 +0000 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 20:43:16.671960502 +0000 2024-04-25T20:48:17.871 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 20:43:16.671960502 +0000 2024-04-25T20:48:17.871 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T20:48:17.933 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T20:48:17.933 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T20:48:17.933 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000179509 s, 2.9 MB/s 2024-04-25T20:48:17.934 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T20:48:17.990 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_3 2024-04-25T20:48:18.044 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T20:48:18.044 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 20:47:54.000823862 +0000 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 20:43:17.013952728 +0000 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 20:43:17.013952728 +0000 2024-04-25T20:48:18.045 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 20:43:17.013952728 +0000 2024-04-25T20:48:18.045 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T20:48:18.107 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T20:48:18.107 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T20:48:18.107 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000177719 s, 2.9 MB/s 2024-04-25T20:48:18.108 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T20:48:18.164 DEBUG:teuthology.orchestra.run.smithi139:> stat /dev/vg_nvme/lv_4 2024-04-25T20:48:18.219 INFO:teuthology.orchestra.run.smithi139.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T20:48:18.219 INFO:teuthology.orchestra.run.smithi139.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:Access: 2024-04-25 20:47:54.001823840 +0000 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:Modify: 2024-04-25 20:43:17.355944955 +0000 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout:Change: 2024-04-25 20:43:17.355944955 +0000 2024-04-25T20:48:18.220 INFO:teuthology.orchestra.run.smithi139.stdout: Birth: 2024-04-25 20:43:17.355944955 +0000 2024-04-25T20:48:18.220 DEBUG:teuthology.orchestra.run.smithi139:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T20:48:18.284 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records in 2024-04-25T20:48:18.284 INFO:teuthology.orchestra.run.smithi139.stderr:1+0 records out 2024-04-25T20:48:18.284 INFO:teuthology.orchestra.run.smithi139.stderr:512 bytes copied, 0.000196325 s, 2.6 MB/s 2024-04-25T20:48:18.285 DEBUG:teuthology.orchestra.run.smithi139:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T20:48:18.342 INFO:tasks.cephadm:Deploying osd.0 on smithi096 with /dev/vg_nvme/lv_4... 2024-04-25T20:48:18.342 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:48:18.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:18.572 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:48:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:48:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:18.694 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:19.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:19 smithi097 ceph-mon[32667]: Deploying daemon mgr.z on smithi139 2024-04-25T20:48:19.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:19 smithi139 ceph-mon[33027]: Deploying daemon mgr.z on smithi139 2024-04-25T20:48:19.627 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:19 smithi096 ceph-mon[29288]: Deploying daemon mgr.z on smithi139 2024-04-25T20:48:20.056 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:19 smithi139 systemd[1]: Starting Ceph mgr.z for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:48:20.314 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:20 smithi139 ceph-mon[33027]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:20.314 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 podman[34869]: 2024-04-25 20:48:19.95870157 +0000 UTC m=+0.018162262 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:48:20.314 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 podman[34869]: 2024-04-25 20:48:20.069642662 +0000 UTC m=+0.129103381 container create e25241a14a5db2ad206f970a8208fd67b105e8bc990d31ea2610fbbe31dad2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T20:48:20.314 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 podman[34869]: 2024-04-25 20:48:20.313795613 +0000 UTC m=+0.373256291 container init e25241a14a5db2ad206f970a8208fd67b105e8bc990d31ea2610fbbe31dad2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-04-25T20:48:20.452 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:48:20.453 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:48:20.453 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T20:48:20.453 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T20:48:20.453 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129419 s, 810 MB/s 2024-04-25T20:48:20.453 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T20:48:20.530 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:20 smithi097 ceph-mon[32667]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:20.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 podman[34869]: 2024-04-25 20:48:20.31717359 +0000 UTC m=+0.376634275 container start e25241a14a5db2ad206f970a8208fd67b105e8bc990d31ea2610fbbe31dad2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T20:48:20.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 ceph-mgr[34889]: -- 172.21.15.139:0/127691845 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e89888a5a0 con 0x55e89885ec00 2024-04-25T20:48:20.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 bash[34869]: e25241a14a5db2ad206f970a8208fd67b105e8bc990d31ea2610fbbe31dad2a3 2024-04-25T20:48:20.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 systemd[1]: Started Ceph mgr.z for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:48:20.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:20.492+0000 7f4912ebf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:48:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:20 smithi096 ceph-mon[29288]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:21.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-04-25T20:48:21.101 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:20.683+0000 7f4912ebf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:48:21.101 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:20.787+0000 7f4912ebf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:48:21.101 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:21.100+0000 7f4912ebf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:48:21.263 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:48:21.372 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:21.186+0000 7f4912ebf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:48:21.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:21.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:48:21.720 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:21.721 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:21.721 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: Deploying daemon mgr.y on smithi097 2024-04-25T20:48:21.721 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:21 smithi096 ceph-mon[29288]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:21.753 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: Deploying daemon mgr.y on smithi097 2024-04-25T20:48:21.754 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:21 smithi097 ceph-mon[32667]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: Deploying daemon mgr.y on smithi097 2024-04-25T20:48:21.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:21 smithi139 ceph-mon[33027]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:22.575 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 systemd[1]: Starting Ceph mgr.y for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:48:22.717 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:22.382+0000 7f4912ebf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:48:22.718 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:22.547+0000 7f4912ebf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:48:22.718 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:22.716+0000 7f4912ebf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:48:22.846 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 podman[34058]: 2024-04-25 20:48:22.477530414 +0000 UTC m=+0.017724561 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:48:22.847 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 podman[34058]: 2024-04-25 20:48:22.575190636 +0000 UTC m=+0.115384779 container create cb310225ad670517c5b1aab87c835d4fc6523c6601b88388d1c661b6b065a44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-25T20:48:22.847 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 podman[34058]: 2024-04-25 20:48:22.819162964 +0000 UTC m=+0.359357097 container init cb310225ad670517c5b1aab87c835d4fc6523c6601b88388d1c661b6b065a44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=) 2024-04-25T20:48:22.847 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 podman[34058]: 2024-04-25 20:48:22.821811648 +0000 UTC m=+0.362005786 container start cb310225ad670517c5b1aab87c835d4fc6523c6601b88388d1c661b6b065a44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T20:48:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 -- 172.21.15.96:0/2881569272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac102aa0 msgr2=0x7f8cac102ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:22.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 --2- 172.21.15.96:0/2881569272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac102aa0 0x7f8cac102ec0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c0099c0 tx=0x7f8c9c009cd0 comp rx=0 tx=0).stop 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 -- 172.21.15.96:0/2881569272 shutdown_connections 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 --2- 172.21.15.96:0/2881569272 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cac104640 0x7f8cac10aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 --2- 172.21.15.96:0/2881569272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac102aa0 0x7f8cac102ec0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 --2- 172.21.15.96:0/2881569272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cac103ca0 0x7f8cac104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.863+0000 7f8cb285e700 1 -- 172.21.15.96:0/2881569272 >> 172.21.15.96:0/2881569272 conn(0x7f8cac0fe040 msgr2=0x7f8cac100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 -- 172.21.15.96:0/2881569272 shutdown_connections 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 -- 172.21.15.96:0/2881569272 wait complete. 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 Processor -- start 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 -- start start 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cac102aa0 0x7f8cac198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:22.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cac103ca0 0x7f8cac199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 0x7f8cac1a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10e510 con 0x7f8cac104640 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10e390 con 0x7f8cac102aa0 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.864+0000 7f8cb285e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8cac10e690 con 0x7f8cac103ca0 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 0x7f8cac1a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cabfff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cac102aa0 0x7f8cac198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 0x7f8cac1a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57226/0 (socket says 172.21.15.96:57226) 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cab7fe700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cac103ca0 0x7f8cac199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 -- 172.21.15.96:0/1424287190 learned_addr learned my addr 172.21.15.96:0/1424287190 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 -- 172.21.15.96:0/1424287190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cac103ca0 msgr2=0x7f8cac199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cac103ca0 0x7f8cac199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 -- 172.21.15.96:0/1424287190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cac102aa0 msgr2=0x7f8cac198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cac102aa0 0x7f8cac198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 -- 172.21.15.96:0/1424287190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cac1a0860 con 0x7f8cac104640 2024-04-25T20:48:22.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb0dfb700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 0x7f8cac1a0190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4007ee0 tx=0x7f8ca400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:22.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400dd20 con 0x7f8cac104640 2024-04-25T20:48:22.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca4004020 con 0x7f8cac104640 2024-04-25T20:48:22.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4015660 con 0x7f8cac104640 2024-04-25T20:48:22.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cac1a0b50 con 0x7f8cac104640 2024-04-25T20:48:22.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.865+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cac1a1060 con 0x7f8cac104640 2024-04-25T20:48:22.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.866+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8ca4015800 con 0x7f8cac104640 2024-04-25T20:48:22.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.866+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cac069730 con 0x7f8cac104640 2024-04-25T20:48:22.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.866+0000 7f8ca97fa700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f8c94039890 0x7f8c9403bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:22.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.866+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8ca404c400 con 0x7f8cac104640 2024-04-25T20:48:22.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.867+0000 7f8cabfff700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f8c94039890 0x7f8c9403bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:22.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.867+0000 7f8cabfff700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f8c94039890 0x7f8c9403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006010 tx=0x7f8c9c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:22.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:22.871+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ca4010070 con 0x7f8cac104640 2024-04-25T20:48:23.004 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:23.003+0000 7f4912ebf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:48:23.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:23.095+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8cac062e80 con 0x7f8c94039890 2024-04-25T20:48:23.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 ceph-mgr[34077]: -- 172.21.15.97:0/3668268403 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556588b205a0 con 0x556588af4c00 2024-04-25T20:48:23.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 bash[34058]: cb310225ad670517c5b1aab87c835d4fc6523c6601b88388d1c661b6b065a44e 2024-04-25T20:48:23.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 systemd[1]: Started Ceph mgr.y for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:48:23.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:22 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:22.974+0000 7f2e2af29200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:48:23.306 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:23.093+0000 7f4912ebf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:48:23.306 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:23.305+0000 7f4912ebf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:48:23.586 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:23.161+0000 7f2e2af29200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:48:23.586 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:23.268+0000 7f2e2af29200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:48:23.586 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:23.584+0000 7f2e2af29200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:48:23.622 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:23.501+0000 7f4912ebf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:48:23.863 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:23.682+0000 7f2e2af29200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:48:24.231 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:24.232 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:24 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:24.143+0000 7f4912ebf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:24.241 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:48:24.242 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:48:24.242 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:24.242 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:24.242 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:24.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:48:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:48:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:24.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:24.872 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:24 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:24.578+0000 7f4912ebf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:48:24.873 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:24 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:24.672+0000 7f4912ebf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:48:25.201 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:24 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:24.873+0000 7f4912ebf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:48:25.201 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:24 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:24.910+0000 7f2e2af29200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:48:25.201 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:25.077+0000 7f2e2af29200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:25.396+0000 7f4912ebf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:25.486+0000 7f4912ebf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.488 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:25.489 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:25.489 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.489 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:25.489 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:25.489 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:25.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:25.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:25.248+0000 7f2e2af29200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:25.532+0000 7f2e2af29200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.534 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:25.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:25.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:25.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-25T20:48:25.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-25T20:48:25.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:25.826 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:25.623+0000 7f2e2af29200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:48:25.826 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:25.824+0000 7f2e2af29200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:48:25.857 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:25.725+0000 7f4912ebf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:48:25.857 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:25.856+0000 7f4912ebf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:48:26.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:26.015+0000 7f2e2af29200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:48:26.122 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:25.950+0000 7f4912ebf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:48:26.122 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:26.062+0000 7f4912ebf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:48:26.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:48:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T20:48:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2952233483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe35739-7b4a-4c48-980e-75a5c703beaa"}]: dispatch 2024-04-25T20:48:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2952233483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fe35739-7b4a-4c48-980e-75a5c703beaa"}]': finished 2024-04-25T20:48:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:48:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2952233483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe35739-7b4a-4c48-980e-75a5c703beaa"}]: dispatch 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2952233483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fe35739-7b4a-4c48-980e-75a5c703beaa"}]': finished 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:48:26.611 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:26.622 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:26.365+0000 7f4912ebf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: Reconfiguring mgr.x (unknown last config time)... 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: Reconfiguring daemon mgr.x on smithi096 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2952233483' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8fe35739-7b4a-4c48-980e-75a5c703beaa"}]: dispatch 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2952233483' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8fe35739-7b4a-4c48-980e-75a5c703beaa"}]': finished 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: osdmap e5: 1 total, 0 up, 1 in 2024-04-25T20:48:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:26.863 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:26 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:26.609+0000 7f2e2af29200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:48:27.208 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:26.954+0000 7f4912ebf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:48:27.208 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:48:27.164+0000 7f4912ebf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:48:27.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:27.213+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7f8ca40148c0 con 0x7f8cac104640 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4214072140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-mon[32667]: Standby manager daemon z started 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:48:27.321 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:27.024+0000 7f2e2af29200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:48:27.322 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:27.119+0000 7f2e2af29200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:48:27.322 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:27.319+0000 7f2e2af29200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:48:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:27 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4214072140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:48:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:27 smithi096 ceph-mon[29288]: Standby manager daemon z started 2024-04-25T20:48:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:27 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:48:27.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:27 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:48:27.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:27 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:48:27.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:27 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:48:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4214072140' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:48:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-mon[33027]: Standby manager daemon z started 2024-04-25T20:48:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:48:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:48:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:48:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:27 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/4138840770' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:48:28.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:27.837+0000 7f2e2af29200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:48:28.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:27.927+0000 7f2e2af29200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:48:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:28 smithi096 ceph-mon[29288]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:28.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:28 smithi096 ceph-mon[29288]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T20:48:28.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:48:28.507 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-mon[32667]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:28.507 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-mon[32667]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T20:48:28.507 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:48:28.508 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:28.165+0000 7f2e2af29200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:48:28.508 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:28.298+0000 7f2e2af29200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:48:28.508 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:28.392+0000 7f2e2af29200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:48:28.508 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:28.505+0000 7f2e2af29200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:48:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:28 smithi139 ceph-mon[33027]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:28 smithi139 ceph-mon[33027]: mgrmap e14: x(active, since 78s), standbys: z 2024-04-25T20:48:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:48:29.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:28.801+0000 7f2e2af29200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:48:29.863 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:29 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:29.400+0000 7f2e2af29200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:48:29.863 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:48:29 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:48:29.614+0000 7f2e2af29200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:48:30.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:30.217+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mon.0 v2:172.21.15.96:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f8ca401dc10 con 0x7f8cac104640 2024-04-25T20:48:30.474 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:30 smithi096 ceph-mon[29288]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:30.474 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:30 smithi096 ceph-mon[29288]: Standby manager daemon y started 2024-04-25T20:48:30.474 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:30 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:48:30.475 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:30 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:48:30.475 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:30 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:48:30.475 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:30 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:48:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:30 smithi097 ceph-mon[32667]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:30 smithi097 ceph-mon[32667]: Standby manager daemon y started 2024-04-25T20:48:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:30 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:48:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:30 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:48:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:30 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:48:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:30 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:48:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:30 smithi139 ceph-mon[33027]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:30 smithi139 ceph-mon[33027]: Standby manager daemon y started 2024-04-25T20:48:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:30 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:48:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:30 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:48:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:30 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:48:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:30 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/456544631' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:48:31.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:31 smithi096 ceph-mon[29288]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T20:48:31.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:31 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:48:31.481 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:31 smithi139 ceph-mon[33027]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T20:48:31.481 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:31 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:48:31.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:31 smithi097 ceph-mon[32667]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-04-25T20:48:31.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:31 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:32.402 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:32.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:32.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:32.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:34.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:34 smithi096 ceph-mon[29288]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:34.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:48:34.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:34 smithi097 ceph-mon[32667]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:48:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:34 smithi139 ceph-mon[33027]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-25T20:48:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:35.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:35 smithi096 ceph-mon[29288]: Deploying daemon osd.0 on smithi096 2024-04-25T20:48:35.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:35 smithi097 ceph-mon[32667]: Deploying daemon osd.0 on smithi096 2024-04-25T20:48:35.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:35 smithi139 ceph-mon[33027]: Deploying daemon osd.0 on smithi096 2024-04-25T20:48:36.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:36 smithi096 ceph-mon[29288]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:36 smithi097 ceph-mon[32667]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:36 smithi139 ceph-mon[33027]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:38 smithi096 ceph-mon[29288]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:38 smithi097 ceph-mon[32667]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:38 smithi139 ceph-mon[33027]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:40.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:40 smithi096 ceph-mon[29288]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:40.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:40 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:40.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:40 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:40.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:40 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:40 smithi097 ceph-mon[32667]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:40 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:40.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:40 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:40.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:40 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:40 smithi139 ceph-mon[33027]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:40 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:40.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:40 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:40.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:40 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:42 smithi096 ceph-mon[29288]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:42.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:42 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:42 smithi097 ceph-mon[32667]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:42 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:42 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:42 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:42.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:42 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:42.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:42 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:42 smithi139 ceph-mon[33027]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:42 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:42 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:42.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:42 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:42.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:42 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:42.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:42 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:43.452 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 0 on host 'smithi096' 2024-04-25T20:48:43.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.450+0000 7f8ca97fa700 1 -- 172.21.15.96:0/1424287190 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8cac062e80 con 0x7f8c94039890 2024-04-25T20:48:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f8c94039890 msgr2=0x7f8c9403bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f8c94039890 0x7f8c9403bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c006010 tx=0x7f8c9c000bc0 comp rx=0 tx=0).stop 2024-04-25T20:48:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 msgr2=0x7f8cac1a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 0x7f8cac1a0190 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4007ee0 tx=0x7f8ca400d3b0 comp rx=0 tx=0).stop 2024-04-25T20:48:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 shutdown_connections 2024-04-25T20:48:43.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f8c94039890 0x7f8c9403bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8cac104640 0x7f8cac1a0190 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8cac102aa0 0x7f8cac198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 --2- 172.21.15.96:0/1424287190 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f8cac103ca0 0x7f8cac199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 >> 172.21.15.96:0/1424287190 conn(0x7f8cac0fe040 msgr2=0x7f8cac107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 shutdown_connections 2024-04-25T20:48:43.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:48:43.453+0000 7f8cb285e700 1 -- 172.21.15.96:0/1424287190 wait complete. 2024-04-25T20:48:44.068 DEBUG:teuthology.orchestra.run.smithi096:osd.0> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.0.service 2024-04-25T20:48:44.070 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_4... 2024-04-25T20:48:44.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:48:44.312 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:48:44.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:44 smithi097 ceph-mon[32667]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:44 smithi097 ceph-mon[32667]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:48:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:44.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:44 smithi096 ceph-mon[29288]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:44.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:44 smithi096 ceph-mon[29288]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:48:44.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:44.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:44.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:44 smithi139 ceph-mon[33027]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:44 smithi139 ceph-mon[33027]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-25T20:48:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:48:45.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:45 smithi096 ceph-mon[29288]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:48:45.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:45 smithi096 ceph-mon[29288]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:48:45.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:45 smithi096 ceph-mon[29288]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:48:45.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:45 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:45.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:45 smithi096 ceph-mon[29288]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:48:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:45 smithi097 ceph-mon[32667]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:48:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:45 smithi097 ceph-mon[32667]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:48:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:45 smithi097 ceph-mon[32667]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:48:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:45 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:45 smithi097 ceph-mon[32667]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:48:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:45 smithi139 ceph-mon[33027]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-25T20:48:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:45 smithi139 ceph-mon[33027]: osdmap e6: 1 total, 0 up, 1 in 2024-04-25T20:48:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:45 smithi139 ceph-mon[33027]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:48:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:45 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:45 smithi139 ceph-mon[33027]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:48:46.335 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:48:46.335 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:48:46.335 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T20:48:46.335 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T20:48:46.335 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126471 s, 829 MB/s 2024-04-25T20:48:46.335 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T20:48:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-mon[29288]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-mon[29288]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:48:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-mon[29288]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' 2024-04-25T20:48:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.500 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0[43881]: 2024-04-25T20:48:46.171+0000 7ff54c878700 -1 osd.0 0 waiting for initial osdmap 2024-04-25T20:48:46.500 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:48:46 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0[43881]: 2024-04-25T20:48:46.367+0000 7ff544663700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:48:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:46 smithi097 ceph-mon[32667]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:46 smithi097 ceph-mon[32667]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:48:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:46 smithi097 ceph-mon[32667]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' 2024-04-25T20:48:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:46 smithi139 ceph-mon[33027]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:46 smithi139 ceph-mon[33027]: osdmap e7: 1 total, 0 up, 1 in 2024-04-25T20:48:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:46 smithi139 ceph-mon[33027]: from='osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354]' entity='osd.0' 2024-04-25T20:48:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:46.920 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2024-04-25T20:48:47.161 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi096 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:47.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.605 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi096 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:47.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi096 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi096 to 17088M 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:48:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:48:47.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:48 smithi097 ceph-mon[32667]: osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] boot 2024-04-25T20:48:48.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:48 smithi097 ceph-mon[32667]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:48:48.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:48 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:48 smithi139 ceph-mon[33027]: osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] boot 2024-04-25T20:48:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:48 smithi139 ceph-mon[33027]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:48:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:48 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:48 smithi096 ceph-mon[29288]: osd.0 [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] boot 2024-04-25T20:48:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:48 smithi096 ceph-mon[29288]: osdmap e8: 1 total, 1 up, 1 in 2024-04-25T20:48:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 -- 172.21.15.97:0/1308810216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 msgr2=0x7fab18104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 --2- 172.21.15.97:0/1308810216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab18104120 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab080099c0 tx=0x7fab08009cd0 comp rx=0 tx=0).stop 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 -- 172.21.15.97:0/1308810216 shutdown_connections 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 --2- 172.21.15.97:0/1308810216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab18104660 0x7fab1810af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 --2- 172.21.15.97:0/1308810216 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab18102ac0 0x7fab18102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 --2- 172.21.15.97:0/1308810216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab18104120 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 -- 172.21.15.97:0/1308810216 >> 172.21.15.97:0/1308810216 conn(0x7fab180fe060 msgr2=0x7fab181004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:48:48.796 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 -- 172.21.15.97:0/1308810216 shutdown_connections 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.793+0000 7fab202e5700 1 -- 172.21.15.97:0/1308810216 wait complete. 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 Processor -- start 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 -- start start 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab18102ac0 0x7fab18198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab181990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab18104660 0x7fab181a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810e5a0 con 0x7fab18102ac0 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810e420 con 0x7fab18103cc0 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab202e5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fab1810e720 con 0x7fab18104660 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab181990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab181990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:39140/0 (socket says 172.21.15.97:39140) 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 -- 172.21.15.97:0/2426574583 learned_addr learned my addr 172.21.15.97:0/2426574583 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T20:48:48.797 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1e081700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab18102ac0 0x7fab18198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1e882700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab18104660 0x7fab181a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 -- 172.21.15.97:0/2426574583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab18104660 msgr2=0x7fab181a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab18104660 0x7fab181a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 -- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab18102ac0 msgr2=0x7fab18198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab18102ac0 0x7fab18198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.794+0000 7fab1d880700 1 -- 172.21.15.97:0/2426574583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab181a0810 con 0x7fab18103cc0 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.795+0000 7fab1d880700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab181990f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab08006010 tx=0x7fab080047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:48.798 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.795+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab08027070 con 0x7fab18103cc0 2024-04-25T20:48:48.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.795+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab080040e0 con 0x7fab18103cc0 2024-04-25T20:48:48.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.795+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab0800f460 con 0x7fab18103cc0 2024-04-25T20:48:48.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.795+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab181a0aa0 con 0x7fab18103cc0 2024-04-25T20:48:48.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.795+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab181a0ef0 con 0x7fab18103cc0 2024-04-25T20:48:48.799 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.796+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab1802df80 con 0x7fab18103cc0 2024-04-25T20:48:48.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.797+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fab0802a070 con 0x7fab18103cc0 2024-04-25T20:48:48.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.797+0000 7fab0f7fe700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab040a3bb0 0x7fab040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:48:48.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.797+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fab080cf6d0 con 0x7fab18103cc0 2024-04-25T20:48:48.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.797+0000 7fab1e081700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab040a3bb0 0x7fab040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:48:48.800 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.798+0000 7fab1e081700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab040a3bb0 0x7fab040a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab18103b20 tx=0x7fab14009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:48:48.802 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:48.802+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab0805eff0 con 0x7fab18103cc0 2024-04-25T20:48:49.007 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:48:49.007+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fab18062e80 con 0x7fab040a3bb0 2024-04-25T20:48:49.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:49 smithi097 ceph-mon[32667]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:48:49.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:49 smithi097 ceph-mon[32667]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:49.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:49 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:48:49.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:49 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:48:49.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:49 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:49.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:49 smithi097 ceph-mon[32667]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:49.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:49 smithi139 ceph-mon[33027]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:48:49.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:49 smithi139 ceph-mon[33027]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:49.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:49 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:48:49.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:49 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:48:49.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:49 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:49.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:49 smithi139 ceph-mon[33027]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:49 smithi096 ceph-mon[29288]: osdmap e9: 1 total, 1 up, 1 in 2024-04-25T20:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:49 smithi096 ceph-mon[29288]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:49.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:49 smithi096 ceph-mon[29288]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:52.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/276270760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]: dispatch 2024-04-25T20:48:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]: dispatch 2024-04-25T20:48:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]': finished 2024-04-25T20:48:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:48:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:48:52.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:52 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/2945279781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:48:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/276270760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]: dispatch 2024-04-25T20:48:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]: dispatch 2024-04-25T20:48:52.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]': finished 2024-04-25T20:48:52.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:48:52.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:48:52.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:52 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/2945279781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/276270760' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]: dispatch 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]: dispatch 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2a2a1f79-9b1b-416c-9797-2417119176d4"}]': finished 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: osdmap e10: 2 total, 1 up, 2 in 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:48:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:52 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/2945279781' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:48:54.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:54 smithi096 ceph-mon[29288]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:54 smithi097 ceph-mon[32667]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:54 smithi139 ceph-mon[33027]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:56.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:56 smithi097 ceph-mon[32667]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:56.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:56 smithi096 ceph-mon[29288]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:56 smithi139 ceph-mon[33027]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:58 smithi096 ceph-mon[29288]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:58 smithi097 ceph-mon[32667]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:58 smithi139 ceph-mon[33027]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:48:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:48:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:48:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:59.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:48:59.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:48:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:48:59.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-25T20:48:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:48:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:00.476 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:00 smithi097 ceph-mon[32667]: Deploying daemon osd.1 on smithi097 2024-04-25T20:49:00.476 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:00 smithi097 ceph-mon[32667]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:00 smithi096 ceph-mon[29288]: Deploying daemon osd.1 on smithi097 2024-04-25T20:49:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:00 smithi096 ceph-mon[29288]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:00 smithi139 ceph-mon[33027]: Deploying daemon osd.1 on smithi097 2024-04-25T20:49:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:00 smithi139 ceph-mon[33027]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:02.493 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:02 smithi097 ceph-mon[32667]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:02 smithi096 ceph-mon[29288]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:02 smithi139 ceph-mon[33027]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:04.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:04 smithi097 ceph-mon[32667]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:04.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:04.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:04 smithi096 ceph-mon[29288]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:04 smithi139 ceph-mon[33027]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:05.576 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:05 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:05.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:05 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:05.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:05 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:06.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:06 smithi097 ceph-mon[32667]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:06 smithi139 ceph-mon[33027]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:06 smithi096 ceph-mon[29288]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:07.871 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.870+0000 7fab0f7fe700 1 -- 172.21.15.97:0/2426574583 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fab18062e80 con 0x7fab040a3bb0 2024-04-25T20:49:07.871 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2024-04-25T20:49:07.873 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab040a3bb0 msgr2=0x7fab040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab040a3bb0 0x7fab040a6070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fab18103b20 tx=0x7fab14009180 comp rx=0 tx=0).stop 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 msgr2=0x7fab181990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab181990f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fab08006010 tx=0x7fab080047b0 comp rx=0 tx=0).stop 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 shutdown_connections 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab040a3bb0 0x7fab040a6070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fab18104660 0x7fab181a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fab18102ac0 0x7fab18198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 --2- 172.21.15.97:0/2426574583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fab18103cc0 0x7fab181990f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 >> 172.21.15.97:0/2426574583 conn(0x7fab180fe060 msgr2=0x7fab18106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 shutdown_connections 2024-04-25T20:49:07.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:49:07.872+0000 7fab202e5700 1 -- 172.21.15.97:0/2426574583 wait complete. 2024-04-25T20:49:08.122 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:08 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:08 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:08.490 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.1.service 2024-04-25T20:49:08.492 INFO:tasks.cephadm:Deploying osd.2 on smithi139 with /dev/vg_nvme/lv_4... 2024-04-25T20:49:08.493 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-25T20:49:08.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:08.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:08.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:08.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:08.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:08 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:08.737 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:49:09.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:09 smithi139 ceph-mon[33027]: from='osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:49:09.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:09 smithi139 ceph-mon[33027]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:49:09.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:09 smithi097 ceph-mon[32667]: from='osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:49:09.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:09 smithi097 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:49:09.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:09 smithi096 ceph-mon[29288]: from='osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:49:09.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:09 smithi096 ceph-mon[29288]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-25T20:49:10.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:10 smithi097 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:49:10.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:10 smithi097 ceph-mon[32667]: from='osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:49:10.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:10 smithi097 ceph-mon[32667]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:49:10.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:10.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:10 smithi097 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:49:10.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:10 smithi097 ceph-mon[32667]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:10.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:10 smithi139 ceph-mon[33027]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:49:10.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:10 smithi139 ceph-mon[33027]: from='osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:49:10.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:10 smithi139 ceph-mon[33027]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:49:10.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:10.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:10 smithi139 ceph-mon[33027]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:49:10.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:10 smithi139 ceph-mon[33027]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:10 smithi096 ceph-mon[29288]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-25T20:49:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:10 smithi096 ceph-mon[29288]: from='osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:49:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:10 smithi096 ceph-mon[29288]: osdmap e11: 2 total, 1 up, 2 in 2024-04-25T20:49:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:10 smithi096 ceph-mon[29288]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:49:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:10 smithi096 ceph-mon[29288]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:10.600 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-25T20:49:10.600 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-25T20:49:10.601 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-04-25T20:49:10.601 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-04-25T20:49:10.601 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124118 s, 845 MB/s 2024-04-25T20:49:10.601 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-04-25T20:49:11.110 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1[37248]: 2024-04-25T20:49:11.048+0000 7f54ab4c6700 -1 osd.1 0 waiting for initial osdmap 2024-04-25T20:49:11.206 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:11 smithi139 ceph-mon[33027]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:49:11.207 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:11 smithi139 ceph-mon[33027]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:49:11.207 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.207 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.207 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:11 smithi139 ceph-mon[33027]: from='osd.1 ' entity='osd.1' 2024-04-25T20:49:11.207 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.234 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_4 2024-04-25T20:49:11.363 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1[37248]: 2024-04-25T20:49:11.179+0000 7f54a2ab0700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:49:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-mon[32667]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:49:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-mon[32667]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:49:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-mon[32667]: from='osd.1 ' entity='osd.1' 2024-04-25T20:49:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.477 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:49:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:11 smithi096 ceph-mon[29288]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:49:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:11 smithi096 ceph-mon[29288]: osdmap e12: 2 total, 1 up, 2 in 2024-04-25T20:49:11.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:11.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:11 smithi096 ceph-mon[29288]: from='osd.1 ' entity='osd.1' 2024-04-25T20:49:11.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: pgmap v63: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 -- 172.21.15.139:0/3040918314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54103cb0 msgr2=0x7fee54104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 --2- 172.21.15.139:0/3040918314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54103cb0 0x7fee54104110 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee3c009990 tx=0x7fee3c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 -- 172.21.15.139:0/3040918314 shutdown_connections 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 --2- 172.21.15.139:0/3040918314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee54104650 0x7fee5410af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 --2- 172.21.15.139:0/3040918314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee54102ab0 0x7fee54102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 --2- 172.21.15.139:0/3040918314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54103cb0 0x7fee54104110 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 -- 172.21.15.139:0/3040918314 >> 172.21.15.139:0/3040918314 conn(0x7fee540fe050 msgr2=0x7fee54100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 -- 172.21.15.139:0/3040918314 shutdown_connections 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 -- 172.21.15.139:0/3040918314 wait complete. 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 Processor -- start 2024-04-25T20:49:13.008 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.001+0000 7fee5931d700 1 -- start start 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 0x7fee54198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee54103cb0 0x7fee54199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee54104650 0x7fee541a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee5410e570 con 0x7fee54104650 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fee5410e3f0 con 0x7fee54103cb0 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee5410e6f0 con 0x7fee54102ab0 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 0x7fee54198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 0x7fee54198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39016/0 (socket says 172.21.15.139:39016) 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 -- 172.21.15.139:0/4048254267 learned_addr learned my addr 172.21.15.139:0/4048254267 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee527fc700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee54103cb0 0x7fee54199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:13.009 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee537fe700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee54104650 0x7fee541a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 -- 172.21.15.139:0/4048254267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee54103cb0 msgr2=0x7fee54199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee54103cb0 0x7fee54199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 -- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee54104650 msgr2=0x7fee541a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee54104650 0x7fee541a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 -- 172.21.15.139:0/4048254267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee541a0870 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee52ffd700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 0x7fee54198c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c970 tx=0x7fee4400cc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44004220 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee541a0b60 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee44007d50 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee44003d00 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.002+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee541a10a0 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.003+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee54069730 con 0x7fee54102ab0 2024-04-25T20:49:13.010 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.004+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fee440078a0 con 0x7fee54102ab0 2024-04-25T20:49:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.005+0000 7fee4bfff700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee400a3bb0 0x7fee400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.005+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7fee440ce720 con 0x7fee54102ab0 2024-04-25T20:49:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.005+0000 7fee527fc700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee400a3bb0 0x7fee400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.006+0000 7fee527fc700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee400a3bb0 0x7fee400a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00b5c0 tx=0x7fee3c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:13.011 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.010+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee44014030 con 0x7fee54102ab0 2024-04-25T20:49:13.218 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:13.217+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fee54062e80 con 0x7fee400a3bb0 2024-04-25T20:49:13.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:13 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi097 2024-04-25T20:49:13.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:13 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T20:49:13.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:13 smithi139 ceph-mon[33027]: osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] boot 2024-04-25T20:49:13.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:13 smithi139 ceph-mon[33027]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:49:13.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:13.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:13 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi097 2024-04-25T20:49:13.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:13 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T20:49:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:13 smithi096 ceph-mon[29288]: osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] boot 2024-04-25T20:49:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:13 smithi096 ceph-mon[29288]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:49:13.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:13 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi097 2024-04-25T20:49:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:13 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi097 to 17088M 2024-04-25T20:49:13.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:13 smithi097 ceph-mon[32667]: osd.1 [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] boot 2024-04-25T20:49:13.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:13 smithi097 ceph-mon[32667]: osdmap e13: 2 total, 2 up, 2 in 2024-04-25T20:49:13.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:49:14.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:14 smithi139 ceph-mon[33027]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:49:14.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:14 smithi139 ceph-mon[33027]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:14.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:14 smithi139 ceph-mon[33027]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:14.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:49:14.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:49:14.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:14.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:14 smithi096 ceph-mon[29288]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:49:14.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:14 smithi096 ceph-mon[29288]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:14.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:14 smithi096 ceph-mon[29288]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:14.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:49:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:49:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:14 smithi097 ceph-mon[32667]: osdmap e14: 2 total, 2 up, 2 in 2024-04-25T20:49:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:14 smithi097 ceph-mon[32667]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:14 smithi097 ceph-mon[32667]: from='client.24172 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:49:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:49:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:16 smithi096 ceph-mon[29288]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:16.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:16 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/3985030579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]: dispatch 2024-04-25T20:49:16.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:16 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]: dispatch 2024-04-25T20:49:16.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:16 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]': finished 2024-04-25T20:49:16.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:16 smithi096 ceph-mon[29288]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:49:16.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:16 smithi097 ceph-mon[32667]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:16 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/3985030579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]: dispatch 2024-04-25T20:49:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:16 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]: dispatch 2024-04-25T20:49:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:16 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]': finished 2024-04-25T20:49:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:16 smithi097 ceph-mon[32667]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:49:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:16 smithi139 ceph-mon[33027]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:16 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/3985030579' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]: dispatch 2024-04-25T20:49:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:16 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]: dispatch 2024-04-25T20:49:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:16 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68afacf2-e118-4883-b8b9-051a6add7851"}]': finished 2024-04-25T20:49:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:16 smithi139 ceph-mon[33027]: osdmap e15: 3 total, 2 up, 3 in 2024-04-25T20:49:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:17.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:17 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/1706168047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:49:17.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:17 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/1706168047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:49:17.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:17 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/1706168047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:49:18.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:18 smithi096 ceph-mon[29288]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:18 smithi097 ceph-mon[32667]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:18 smithi139 ceph-mon[33027]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:20 smithi096 ceph-mon[29288]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:20.505 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:20 smithi139 ceph-mon[33027]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:20 smithi097 ceph-mon[32667]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:22.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:22 smithi139 ceph-mon[33027]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:22 smithi096 ceph-mon[29288]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:22.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:22 smithi097 ceph-mon[32667]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:24 smithi139 ceph-mon[33027]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:24 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:49:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:24 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:24 smithi096 ceph-mon[29288]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:49:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:24 smithi097 ceph-mon[32667]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:24 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-25T20:49:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:24 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:25.371 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:25 smithi139 ceph-mon[33027]: Deploying daemon osd.2 on smithi139 2024-04-25T20:49:25.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:25 smithi097 ceph-mon[32667]: Deploying daemon osd.2 on smithi139 2024-04-25T20:49:25.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:25 smithi096 ceph-mon[29288]: Deploying daemon osd.2 on smithi139 2024-04-25T20:49:26.585 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:26 smithi139 ceph-mon[33027]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:26 smithi097 ceph-mon[32667]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:26 smithi096 ceph-mon[29288]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:28.537 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:28 smithi139 ceph-mon[33027]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:28 smithi097 ceph-mon[32667]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:28 smithi096 ceph-mon[29288]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:30.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:30.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:29 smithi096 ceph-mon[29288]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:30.344 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:30.344 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:30.344 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:30.344 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:29 smithi139 ceph-mon[33027]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:29 smithi097 ceph-mon[32667]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:32.282 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 2 on host 'smithi139' 2024-04-25T20:49:32.282 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.281+0000 7fee4bfff700 1 -- 172.21.15.139:0/4048254267 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fee54062e80 con 0x7fee400a3bb0 2024-04-25T20:49:32.284 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee400a3bb0 msgr2=0x7fee400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee400a3bb0 0x7fee400a6070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00b5c0 tx=0x7fee3c005fb0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 msgr2=0x7fee54198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 0x7fee54198c20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fee4400c970 tx=0x7fee4400cc80 comp rx=0 tx=0).stop 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 shutdown_connections 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee400a3bb0 0x7fee400a6070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee54104650 0x7fee541a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee54102ab0 0x7fee54198c20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 --2- 172.21.15.139:0/4048254267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee54103cb0 0x7fee54199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.283+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 >> 172.21.15.139:0/4048254267 conn(0x7fee540fe050 msgr2=0x7fee54107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.284+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 shutdown_connections 2024-04-25T20:49:32.285 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:49:32.284+0000 7fee5931d700 1 -- 172.21.15.139:0/4048254267 wait complete. 2024-04-25T20:49:32.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:32 smithi096 ceph-mon[29288]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:32 smithi097 ceph-mon[32667]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:32 smithi139 ceph-mon[33027]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:32.948 DEBUG:teuthology.orchestra.run.smithi139:osd.2> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.2.service 2024-04-25T20:49:32.950 INFO:tasks.cephadm:Deploying osd.3 on smithi096 with /dev/vg_nvme/lv_3... 2024-04-25T20:49:32.951 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:49:33.195 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:49:33.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:33 smithi096 ceph-mon[29288]: from='osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:49:33.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:33 smithi096 ceph-mon[29288]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:49:33.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:33 smithi097 ceph-mon[32667]: from='osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:49:33.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:33 smithi097 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:49:33.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:33 smithi139 ceph-mon[33027]: from='osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:49:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:33 smithi139 ceph-mon[33027]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-25T20:49:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-mon[33027]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-mon[33027]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:49:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-mon[33027]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:49:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-mon[33027]: from='osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:49:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-mon[33027]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:49:34.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2[38159]: 2024-04-25T20:49:34.301+0000 7f43a5fdb700 -1 osd.2 0 waiting for initial osdmap 2024-04-25T20:49:34.374 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2[38159]: 2024-04-25T20:49:34.305+0000 7f43a1dce700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:49:34.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:34 smithi096 ceph-mon[29288]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:34.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:34 smithi096 ceph-mon[29288]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:49:34.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:34 smithi096 ceph-mon[29288]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:49:34.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:34.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:34 smithi096 ceph-mon[29288]: from='osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:49:34.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:34 smithi096 ceph-mon[29288]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:49:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:34 smithi097 ceph-mon[32667]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:34 smithi097 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-25T20:49:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:34 smithi097 ceph-mon[32667]: osdmap e16: 3 total, 2 up, 3 in 2024-04-25T20:49:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:34 smithi097 ceph-mon[32667]: from='osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:49:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:34 smithi097 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:49:35.064 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:49:35.064 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:49:35.064 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T20:49:35.064 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T20:49:35.064 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0122256 s, 858 MB/s 2024-04-25T20:49:35.064 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T20:49:35.422 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:35 smithi096 ceph-mon[29288]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:49:35.422 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:35 smithi096 ceph-mon[29288]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:49:35.422 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:35.422 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:35.502 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:35 smithi139 ceph-mon[33027]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:49:35.502 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:35 smithi139 ceph-mon[33027]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:49:35.502 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:35.502 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:35.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:35 smithi097 ceph-mon[32667]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:49:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:35 smithi097 ceph-mon[32667]: osdmap e17: 3 total, 2 up, 3 in 2024-04-25T20:49:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:35.764 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-04-25T20:49:36.005 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] boot 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:49:36.459 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:36.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:36 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] boot 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:36.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:36.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:36.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:36.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:36.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:36 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: pgmap v80: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: osd.2 [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] boot 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: osdmap e18: 3 total, 3 up, 3 in 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:49:36.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:36 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:37.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:37 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi139 2024-04-25T20:49:37.456 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:37 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi139 to 17088M 2024-04-25T20:49:37.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:37 smithi096 ceph-mon[29288]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T20:49:37.457 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:49:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:37 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi139 2024-04-25T20:49:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:37 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi139 to 17088M 2024-04-25T20:49:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:37 smithi097 ceph-mon[32667]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T20:49:37.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:49:37.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:37 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi139 2024-04-25T20:49:37.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:37 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi139 to 17088M 2024-04-25T20:49:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:37 smithi139 ceph-mon[33027]: osdmap e19: 3 total, 3 up, 3 in 2024-04-25T20:49:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 -- 172.21.15.96:0/1579538836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc103df0 msgr2=0x7f67dc104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1579538836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc103df0 0x7f67dc104250 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f67c40099c0 tx=0x7f67c4009cd0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 -- 172.21.15.96:0/1579538836 shutdown_connections 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1579538836 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67dc104790 0x7f67dc10b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1579538836 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67dc100d70 0x7f67dc101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1579538836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc103df0 0x7f67dc104250 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 -- 172.21.15.96:0/1579538836 >> 172.21.15.96:0/1579538836 conn(0x7f67dc0fc6d0 msgr2=0x7f67dc0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 -- 172.21.15.96:0/1579538836 shutdown_connections 2024-04-25T20:49:37.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 -- 172.21.15.96:0/1579538836 wait complete. 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.684+0000 7f67e14cf700 1 Processor -- start 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 -- start start 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 0x7f67dc198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67dc103df0 0x7f67dc1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67dc104790 0x7f67dc1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc10e5a0 con 0x7f67dc100d70 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc10e420 con 0x7f67dc104790 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67e14cf700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67dc10e720 con 0x7f67dc103df0 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 0x7f67dc198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 0x7f67dc198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44276/0 (socket says 172.21.15.96:44276) 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 -- 172.21.15.96:0/1731109116 learned_addr learned my addr 172.21.15.96:0/1731109116 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67db7fe700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67dc104790 0x7f67dc1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67d27fc700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67dc103df0 0x7f67dc1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 -- 172.21.15.96:0/1731109116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67dc103df0 msgr2=0x7f67dc1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67dc103df0 0x7f67dc1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 -- 172.21.15.96:0/1731109116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67dc104790 msgr2=0x7f67dc1a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67dc104790 0x7f67dc1a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 -- 172.21.15.96:0/1731109116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67dc1a08b0 con 0x7f67dc100d70 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67daffd700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 0x7f67dc198c60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00b500 tx=0x7f67cc00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.685+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc015020 con 0x7f67dc100d70 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.686+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67dc1a0ba0 con 0x7f67dc100d70 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.686+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67cc011ce0 con 0x7f67dc100d70 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.686+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67cc010d20 con 0x7f67dc100d70 2024-04-25T20:49:37.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.686+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67dc1a10e0 con 0x7f67dc100d70 2024-04-25T20:49:37.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.686+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67dc069730 con 0x7f67dc100d70 2024-04-25T20:49:37.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.688+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f67cc018020 con 0x7f67dc100d70 2024-04-25T20:49:37.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.689+0000 7f67d8ff9700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f67c80a3bb0 0x7f67c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:49:37.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.689+0000 7f67d27fc700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f67c80a3bb0 0x7f67c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:49:37.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.689+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7f67cc0cea70 con 0x7f67dc100d70 2024-04-25T20:49:37.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.689+0000 7f67d27fc700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f67c80a3bb0 0x7f67c80a6070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67c4000c00 tx=0x7f67c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:49:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.693+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67cc01f020 con 0x7f67dc100d70 2024-04-25T20:49:37.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:37.903+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f67dc062e80 con 0x7f67c80a3bb0 2024-04-25T20:49:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T20:49:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T20:49:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:49:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:49:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:49:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:38.622 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:38.622 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T20:49:38.622 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T20:49:38.622 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:49:38.622 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:49:38.622 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: pgmap v83: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: osdmap e20: 3 total, 3 up, 3 in 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:49:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:39.249 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48277]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48277]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48277]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48277]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48280]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48280]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48280]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.250 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 20:49:38 smithi096 sudo[48280]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:49:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48286]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48286]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48289]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:49:39.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48289]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.251 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 sudo[48289]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40954]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40954]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40954]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40954]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40957]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40957]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40957]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.310 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40957]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40948]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40948]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40948]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40948]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40951]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40951]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40951]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.311 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 20:49:39 smithi097 sudo[40951]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41757]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41757]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41757]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41757]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41760]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41760]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41760]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.373 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41760]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41763]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41763]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41763]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41763]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41766]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41766]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41766]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 sudo[41766]: pam_unix(sudo:session): session closed for user root 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T20:49:39.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.375 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.376 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:39 smithi139 ceph-mon[33027]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:39 smithi097 ceph-mon[32667]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: osdmap e21: 3 total, 3 up, 3 in 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:39.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:49:39.688 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:39 smithi096 ceph-mon[29288]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-25T20:49:40.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:40.157+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67cc010380 con 0x7f67dc100d70 2024-04-25T20:49:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:40 smithi097 ceph-mon[32667]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:40 smithi097 ceph-mon[32667]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T20:49:40.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:40 smithi097 ceph-mon[32667]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T20:49:40.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:40 smithi139 ceph-mon[33027]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:40.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:40 smithi139 ceph-mon[33027]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T20:49:40.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:40 smithi139 ceph-mon[33027]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T20:49:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:40 smithi096 ceph-mon[29288]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:40 smithi096 ceph-mon[29288]: osdmap e22: 3 total, 3 up, 3 in 2024-04-25T20:49:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:40 smithi096 ceph-mon[29288]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-25T20:49:41.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:41 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2731901089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf6c36f-7417-4b70-961e-a1712eb00fa1"}]: dispatch 2024-04-25T20:49:41.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:41 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2731901089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf6c36f-7417-4b70-961e-a1712eb00fa1"}]': finished 2024-04-25T20:49:41.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:41 smithi097 ceph-mon[32667]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T20:49:41.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:49:41.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:41 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2552815419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:49:41.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:41 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2731901089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf6c36f-7417-4b70-961e-a1712eb00fa1"}]: dispatch 2024-04-25T20:49:41.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:41 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2731901089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf6c36f-7417-4b70-961e-a1712eb00fa1"}]': finished 2024-04-25T20:49:41.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:41 smithi139 ceph-mon[33027]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T20:49:41.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:49:41.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:41 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2552815419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:49:41.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:41 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2731901089' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9cf6c36f-7417-4b70-961e-a1712eb00fa1"}]: dispatch 2024-04-25T20:49:41.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:41 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2731901089' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9cf6c36f-7417-4b70-961e-a1712eb00fa1"}]': finished 2024-04-25T20:49:41.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:41 smithi096 ceph-mon[29288]: osdmap e23: 4 total, 3 up, 4 in 2024-04-25T20:49:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:49:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:41 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2552815419' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:49:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:42 smithi097 ceph-mon[32667]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:42 smithi139 ceph-mon[33027]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:42 smithi096 ceph-mon[29288]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:44.572 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:44 smithi096 ceph-mon[29288]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:44 smithi097 ceph-mon[32667]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:44 smithi139 ceph-mon[33027]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:46 smithi096 ceph-mon[29288]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:46 smithi097 ceph-mon[32667]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:46 smithi139 ceph-mon[33027]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:48 smithi096 ceph-mon[29288]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T20:49:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:48 smithi097 ceph-mon[32667]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:48 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T20:49:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:48 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:48 smithi139 ceph-mon[33027]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:48 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-25T20:49:48.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:48 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:49:49.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:49 smithi096 ceph-mon[29288]: Deploying daemon osd.3 on smithi096 2024-04-25T20:49:49.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:49 smithi097 ceph-mon[32667]: Deploying daemon osd.3 on smithi096 2024-04-25T20:49:49.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:49 smithi139 ceph-mon[33027]: Deploying daemon osd.3 on smithi096 2024-04-25T20:49:50.365 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:50 smithi096 ceph-mon[29288]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:50 smithi097 ceph-mon[32667]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:50 smithi139 ceph-mon[33027]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:51.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:51 smithi096 ceph-mon[29288]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:51.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:51 smithi097 ceph-mon[32667]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:51 smithi139 ceph-mon[33027]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:54.387 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:54 smithi096 ceph-mon[29288]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:54.387 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:54 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:54.387 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:54 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:54.387 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:54 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:54 smithi097 ceph-mon[32667]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:54 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:54 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:54 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:54 smithi139 ceph-mon[33027]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:54 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:54 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:54 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:49:56.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:56 smithi096 ceph-mon[29288]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:56 smithi097 ceph-mon[32667]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:56 smithi139 ceph-mon[33027]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:57.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.172+0000 7f67d8ff9700 1 -- 172.21.15.96:0/1731109116 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67dc062e80 con 0x7f67c80a3bb0 2024-04-25T20:49:57.174 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 3 on host 'smithi096' 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.174+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f67c80a3bb0 msgr2=0x7f67c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.174+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f67c80a3bb0 0x7f67c80a6070 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f67c4000c00 tx=0x7f67c4005fb0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 msgr2=0x7f67dc198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 0x7f67dc198c60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f67cc00b500 tx=0x7f67cc00b810 comp rx=0 tx=0).stop 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 shutdown_connections 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f67c80a3bb0 0x7f67c80a6070 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67dc104790 0x7f67dc1a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67dc100d70 0x7f67dc198c60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 --2- 172.21.15.96:0/1731109116 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67dc103df0 0x7f67dc1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 >> 172.21.15.96:0/1731109116 conn(0x7f67dc0fc6d0 msgr2=0x7f67dc107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 shutdown_connections 2024-04-25T20:49:57.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:49:57.175+0000 7f67e14cf700 1 -- 172.21.15.96:0/1731109116 wait complete. 2024-04-25T20:49:57.874 DEBUG:teuthology.orchestra.run.smithi096:osd.3> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.3.service 2024-04-25T20:49:57.877 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2024-04-25T20:49:57.877 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:49:58.124 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:49:58.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:58 smithi097 ceph-mon[32667]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:58.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:58 smithi097 ceph-mon[32667]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T20:49:58.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.247 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:58 smithi096 ceph-mon[29288]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:58 smithi096 ceph-mon[29288]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T20:49:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:58 smithi139 ceph-mon[33027]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:49:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:58 smithi139 ceph-mon[33027]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-25T20:49:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:49:59.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:59 smithi096 ceph-mon[29288]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T20:49:59.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:59 smithi096 ceph-mon[29288]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T20:49:59.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:59 smithi096 ceph-mon[29288]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:49:59.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:49:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:49:59.250 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 20:49:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T20:49:59.143+0000 7fa21c791700 -1 osd.3 0 waiting for initial osdmap 2024-04-25T20:49:59.250 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 20:49:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T20:49:59.149+0000 7fa218584700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:49:59.437 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:59 smithi097 ceph-mon[32667]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T20:49:59.437 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:59 smithi097 ceph-mon[32667]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T20:49:59.437 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:59 smithi097 ceph-mon[32667]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:49:59.437 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:49:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:49:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:59 smithi139 ceph-mon[33027]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-25T20:49:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:59 smithi139 ceph-mon[33027]: osdmap e24: 4 total, 3 up, 4 in 2024-04-25T20:49:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:59 smithi139 ceph-mon[33027]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:49:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:49:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.097 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:50:00.097 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:50:00.097 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T20:50:00.098 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T20:50:00.098 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124268 s, 844 MB/s 2024-04-25T20:50:00.098 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T20:50:00.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] boot 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:00 smithi097 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T20:50:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] boot 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:00 smithi096 ceph-mon[29288]: overall HEALTH_OK 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: from='osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: osdmap e25: 4 total, 3 up, 4 in 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: osd.3 [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] boot 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: osdmap e26: 4 total, 4 up, 4 in 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:50:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:00 smithi139 ceph-mon[33027]: overall HEALTH_OK 2024-04-25T20:50:00.710 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2024-04-25T20:50:00.944 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:50:01.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:01 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:50:01.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:01 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:50:01.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:01 smithi096 ceph-mon[29288]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T20:50:01.325 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:01 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:50:01.326 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:01 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:50:01.326 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:01 smithi097 ceph-mon[32667]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T20:50:01.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:01 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:50:01.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:01 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:50:01.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:01 smithi139 ceph-mon[33027]: osdmap e27: 4 total, 4 up, 4 in 2024-04-25T20:50:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.482 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.483 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.484 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:02 smithi097 ceph-mon[32667]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T20:50:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 -- 172.21.15.97:0/186439262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc105f00 msgr2=0x7f03dc1082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 --2- 172.21.15.97:0/186439262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc105f00 0x7f03dc1082f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009990 tx=0x7f03cc009ca0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 -- 172.21.15.97:0/186439262 shutdown_connections 2024-04-25T20:50:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 --2- 172.21.15.97:0/186439262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03dc108830 0x7f03dc10ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.495 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 --2- 172.21.15.97:0/186439262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03dc1035d0 0x7f03dc1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 --2- 172.21.15.97:0/186439262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc105f00 0x7f03dc1082f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 -- 172.21.15.97:0/186439262 >> 172.21.15.97:0/186439262 conn(0x7f03dc0fcf40 msgr2=0x7f03dc0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 -- 172.21.15.97:0/186439262 shutdown_connections 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 -- 172.21.15.97:0/186439262 wait complete. 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.493+0000 7f03e3777700 1 Processor -- start 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 -- start start 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 0x7f03dc19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03dc105f00 0x7f03dc19b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03dc108830 0x7f03dc1a2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10e5a0 con 0x7f03dc105f00 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10e420 con 0x7f03dc1035d0 2024-04-25T20:50:02.496 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e3777700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03dc10e720 con 0x7f03dc108830 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 0x7f03dc19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 0x7f03dc19ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:59402/0 (socket says 172.21.15.97:59402) 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 -- 172.21.15.97:0/4073921296 learned_addr learned my addr 172.21.15.97:0/4073921296 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1d14700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03dc108830 0x7f03dc1a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e0d12700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03dc105f00 0x7f03dc19b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 -- 172.21.15.97:0/4073921296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03dc108830 msgr2=0x7f03dc1a2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03dc108830 0x7f03dc1a2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 -- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03dc105f00 msgr2=0x7f03dc19b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03dc105f00 0x7f03dc19b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 -- 172.21.15.97:0/4073921296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03dc1a2aa0 con 0x7f03dc1035d0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.494+0000 7f03e1513700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 0x7f03dc19ae40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03d800d950 tx=0x7f03d800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.495+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d8016070 con 0x7f03dc1035d0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.495+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03dc1a2d90 con 0x7f03dc1035d0 2024-04-25T20:50:02.497 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.495+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03d800fd40 con 0x7f03dc1035d0 2024-04-25T20:50:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.495+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d800e3b0 con 0x7f03dc1035d0 2024-04-25T20:50:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.495+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03dc1a3270 con 0x7f03dc1035d0 2024-04-25T20:50:02.498 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.496+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03dc069730 con 0x7f03dc1035d0 2024-04-25T20:50:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.497+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f03d8019070 con 0x7f03dc1035d0 2024-04-25T20:50:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.498+0000 7f03d27fc700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03c80a3c40 0x7f03c80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.498+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f03d80cebf0 con 0x7f03dc1035d0 2024-04-25T20:50:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.498+0000 7f03e0d12700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03c80a3c40 0x7f03c80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:02.499 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.499+0000 7f03e0d12700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03c80a3c40 0x7f03c80a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00b5c0 tx=0x7f03cc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:02 smithi096 ceph-mon[29288]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T20:50:02.502 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.502+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03d809c7d0 con 0x7f03dc1035d0 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:02.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:02.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:02.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:02.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:02.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:02 smithi139 ceph-mon[33027]: osdmap e28: 4 total, 4 up, 4 in 2024-04-25T20:50:02.707 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:02.707+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f03dc062e80 con 0x7f03c80a3c40 2024-04-25T20:50:03.438 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:03 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi096 2024-04-25T20:50:03.438 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:03 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T20:50:03.438 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:03.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:03.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:03.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:03 smithi097 ceph-mon[32667]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T20:50:03.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:03 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi096 2024-04-25T20:50:03.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:03 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T20:50:03.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:03.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:03.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:03.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:03 smithi096 ceph-mon[29288]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T20:50:03.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:03 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi096 2024-04-25T20:50:03.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:03 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi096 to 8544M 2024-04-25T20:50:03.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:03.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:03.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:03.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:03 smithi139 ceph-mon[33027]: osdmap e29: 4 total, 4 up, 4 in 2024-04-25T20:50:04.263 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:04 smithi097 ceph-mon[32667]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:04.264 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:04 smithi097 ceph-mon[32667]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:50:04.264 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:04 smithi097 ceph-mon[32667]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T20:50:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:04 smithi096 ceph-mon[29288]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:04 smithi096 ceph-mon[29288]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:50:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:04 smithi096 ceph-mon[29288]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T20:50:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:04 smithi139 ceph-mon[33027]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:04 smithi139 ceph-mon[33027]: pgmap v106: 1 pgs: 1 peering; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:50:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:04 smithi139 ceph-mon[33027]: osdmap e30: 4 total, 4 up, 4 in 2024-04-25T20:50:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:50:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/422595776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]: dispatch 2024-04-25T20:50:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]: dispatch 2024-04-25T20:50:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]': finished 2024-04-25T20:50:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T20:50:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:06 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/1739192985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/422595776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]: dispatch 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]: dispatch 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]': finished 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:06.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:06 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/1739192985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: pgmap v108: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:50:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/422595776' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]: dispatch 2024-04-25T20:50:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]: dispatch 2024-04-25T20:50:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd60dbbf-deec-4429-b64c-dc7238e7bb49"}]': finished 2024-04-25T20:50:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: osdmap e31: 5 total, 4 up, 5 in 2024-04-25T20:50:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:06 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/1739192985' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:08 smithi097 ceph-mon[32667]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T20:50:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:08 smithi139 ceph-mon[33027]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T20:50:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:08 smithi096 ceph-mon[29288]: pgmap v110: 1 pgs: 1 peering; 577 KiB data, 121 MiB used, 357 GiB / 358 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T20:50:10.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:10 smithi097 ceph-mon[32667]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T20:50:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:10 smithi139 ceph-mon[33027]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T20:50:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:10 smithi096 ceph-mon[29288]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T20:50:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:12 smithi097 ceph-mon[32667]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:12 smithi139 ceph-mon[33027]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:12 smithi096 ceph-mon[29288]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:13.548 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T20:50:13.548 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:13.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T20:50:13.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:13.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-25T20:50:13.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:14 smithi097 ceph-mon[32667]: Deploying daemon osd.4 on smithi097 2024-04-25T20:50:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:14 smithi097 ceph-mon[32667]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:14 smithi139 ceph-mon[33027]: Deploying daemon osd.4 on smithi097 2024-04-25T20:50:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:14 smithi139 ceph-mon[33027]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:14 smithi096 ceph-mon[29288]: Deploying daemon osd.4 on smithi097 2024-04-25T20:50:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:14 smithi096 ceph-mon[29288]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:16.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:16 smithi097 ceph-mon[32667]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:16 smithi139 ceph-mon[33027]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:16 smithi096 ceph-mon[29288]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:18 smithi097 ceph-mon[32667]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:18 smithi139 ceph-mon[33027]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:18 smithi096 ceph-mon[29288]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:19.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:19 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:19.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:19 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:19.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:19 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:20.374 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:20 smithi097 ceph-mon[32667]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:20 smithi139 ceph-mon[33027]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:20 smithi096 ceph-mon[29288]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:21.873 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2024-04-25T20:50:21.874 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.872+0000 7f03d27fc700 1 -- 172.21.15.97:0/4073921296 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03dc062e80 con 0x7f03c80a3c40 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03c80a3c40 msgr2=0x7f03c80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03c80a3c40 0x7f03c80a6100 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00b5c0 tx=0x7f03cc005fb0 comp rx=0 tx=0).stop 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 msgr2=0x7f03dc19ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 0x7f03dc19ae40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f03d800d950 tx=0x7f03d800dc60 comp rx=0 tx=0).stop 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 shutdown_connections 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03c80a3c40 0x7f03c80a6100 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03dc108830 0x7f03dc1a2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:21.876 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03dc1035d0 0x7f03dc19ae40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:21.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 --2- 172.21.15.97:0/4073921296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03dc105f00 0x7f03dc19b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:21.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.874+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 >> 172.21.15.97:0/4073921296 conn(0x7f03dc0fcf40 msgr2=0x7f03dc102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:21.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.875+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 shutdown_connections 2024-04-25T20:50:21.877 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:50:21.875+0000 7f03e3777700 1 -- 172.21.15.97:0/4073921296 wait complete. 2024-04-25T20:50:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:22.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:22.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:22.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:22.468 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.4.service 2024-04-25T20:50:22.470 INFO:tasks.cephadm:Deploying osd.5 on smithi139 with /dev/vg_nvme/lv_3... 2024-04-25T20:50:22.470 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-25T20:50:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:22.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:22.714 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:50:23.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:23 smithi139 ceph-mon[33027]: from='osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:50:23.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:23 smithi139 ceph-mon[33027]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:50:23.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:23 smithi097 ceph-mon[32667]: from='osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:50:23.298 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:23 smithi097 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:50:23.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:23 smithi096 ceph-mon[29288]: from='osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:50:23.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:23 smithi096 ceph-mon[29288]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-25T20:50:24.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T20:50:24.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-mon[32667]: from='osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:50:24.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-mon[32667]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T20:50:24.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:24.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:50:24.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-mon[32667]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:24.114 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4[43773]: 2024-04-25T20:50:24.053+0000 7fdb36cc4700 -1 osd.4 0 waiting for initial osdmap 2024-04-25T20:50:24.115 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 20:50:24 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4[43773]: 2024-04-25T20:50:24.060+0000 7fdb32ab7700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:50:24.298 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:24 smithi139 ceph-mon[33027]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T20:50:24.299 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:24 smithi139 ceph-mon[33027]: from='osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:50:24.299 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:24 smithi139 ceph-mon[33027]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T20:50:24.299 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:24 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:24.299 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:24 smithi139 ceph-mon[33027]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:50:24.299 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:24 smithi139 ceph-mon[33027]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:24 smithi096 ceph-mon[29288]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-25T20:50:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:24 smithi096 ceph-mon[29288]: from='osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:50:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:24 smithi096 ceph-mon[29288]: osdmap e32: 5 total, 4 up, 5 in 2024-04-25T20:50:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:24 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:24 smithi096 ceph-mon[29288]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:50:24.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:24 smithi096 ceph-mon[29288]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-25T20:50:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-25T20:50:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-25T20:50:24.635 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-04-25T20:50:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-04-25T20:50:24.635 INFO:teuthology.orchestra.run.smithi139.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0125232 s, 837 MB/s 2024-04-25T20:50:24.636 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 2024-04-25T20:50:24.636 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-04-25T20:50:25.252 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:25 smithi139 ceph-mon[33027]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:50:25.252 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:25 smithi139 ceph-mon[33027]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T20:50:25.252 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:25.252 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:25.279 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_3 2024-04-25T20:50:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:25 smithi097 ceph-mon[32667]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:50:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:25 smithi097 ceph-mon[32667]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T20:50:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:25.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:25 smithi096 ceph-mon[29288]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:50:25.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:25 smithi096 ceph-mon[29288]: osdmap e33: 5 total, 4 up, 5 in 2024-04-25T20:50:25.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:25.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:25.522 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] boot 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:26.341 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:26.342 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:50:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:50:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] boot 2024-04-25T20:50:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T20:50:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:26.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: osd.4 [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] boot 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: osdmap e34: 5 total, 5 up, 5 in 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 -- 172.21.15.139:0/3811804722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 msgr2=0x7f2784109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 --2- 172.21.15.139:0/3811804722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f2784109df0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f277c00b0a0 tx=0x7f277c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:50:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 -- 172.21.15.139:0/3811804722 shutdown_connections 2024-04-25T20:50:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 --2- 172.21.15.139:0/3811804722 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f2784109df0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:27.100 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 --2- 172.21.15.139:0/3811804722 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27841019a0 0x7f2784101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 --2- 172.21.15.139:0/3811804722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2784102ba0 0x7f2784103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 -- 172.21.15.139:0/3811804722 >> 172.21.15.139:0/3811804722 conn(0x7f27840fcf40 msgr2=0x7f27840ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 -- 172.21.15.139:0/3811804722 shutdown_connections 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.096+0000 7f278af78700 1 -- 172.21.15.139:0/3811804722 wait complete. 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 Processor -- start 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 -- start start 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27841019a0 0x7f2784197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784102ba0 0x7f2784197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f278419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f278410d440 con 0x7f27841019a0 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f278410d2c0 con 0x7f2784102ba0 2024-04-25T20:50:27.101 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f278af78700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f278410d5c0 con 0x7f2784103540 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f278419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f278419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:59962/0 (socket says 172.21.15.139:59962) 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 -- 172.21.15.139:0/3912976790 learned_addr learned my addr 172.21.15.139:0/3912976790 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2788d14700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27841019a0 0x7f2784197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2783fff700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784102ba0 0x7f2784197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 -- 172.21.15.139:0/3912976790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784102ba0 msgr2=0x7f2784197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784102ba0 0x7f2784197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 -- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27841019a0 msgr2=0x7f2784197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27841019a0 0x7f2784197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2789515700 1 -- 172.21.15.139:0/3912976790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278419f690 con 0x7f2784103540 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.097+0000 7f2783fff700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784102ba0 0x7f2784197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:50:27.102 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.098+0000 7f2789515700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f278419eff0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f277c009610 tx=0x7f277c004ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.098+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c021070 con 0x7f2784103540 2024-04-25T20:50:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.098+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f277c007db0 con 0x7f2784103540 2024-04-25T20:50:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.098+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f277c01d450 con 0x7f2784103540 2024-04-25T20:50:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.098+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278419f920 con 0x7f2784103540 2024-04-25T20:50:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.098+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278419fdd0 con 0x7f2784103540 2024-04-25T20:50:27.103 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.099+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2784069730 con 0x7f2784103540 2024-04-25T20:50:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.100+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f277c004020 con 0x7f2784103540 2024-04-25T20:50:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.100+0000 7f2781ffb700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27740a3c40 0x7f27740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.101+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f277c00f020 con 0x7f2784103540 2024-04-25T20:50:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.101+0000 7f2788d14700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27740a3c40 0x7f27740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:27.104 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.102+0000 7f2788d14700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27740a3c40 0x7f27740a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2778005950 tx=0x7f27780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:27.106 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.105+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f277c02a020 con 0x7f2784103540 2024-04-25T20:50:27.312 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:27.311+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f2784062f40 con 0x7f27740a3c40 2024-04-25T20:50:27.312 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:27 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi097 2024-04-25T20:50:27.312 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:27 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T20:50:27.312 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:27 smithi139 ceph-mon[33027]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T20:50:27.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:27 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi097 2024-04-25T20:50:27.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:27 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T20:50:27.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:27 smithi097 ceph-mon[32667]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T20:50:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:27 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi097 2024-04-25T20:50:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:27 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi097 to 8544M 2024-04-25T20:50:27.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:27 smithi096 ceph-mon[29288]: osdmap e35: 5 total, 5 up, 5 in 2024-04-25T20:50:28.280 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:28 smithi139 ceph-mon[33027]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:28.280 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:28 smithi139 ceph-mon[33027]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:28.280 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:28.280 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:28.280 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:28.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:28 smithi097 ceph-mon[32667]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:28.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:28 smithi097 ceph-mon[32667]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:28.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:28.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:28.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:28 smithi096 ceph-mon[29288]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:28 smithi096 ceph-mon[29288]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:28.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:28.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:30 smithi096 ceph-mon[29288]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:30 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/2748120572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]: dispatch 2024-04-25T20:50:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:30 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]: dispatch 2024-04-25T20:50:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:30 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]': finished 2024-04-25T20:50:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:30 smithi096 ceph-mon[29288]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T20:50:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:30 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:30 smithi097 ceph-mon[32667]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:30 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/2748120572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]: dispatch 2024-04-25T20:50:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:30 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]: dispatch 2024-04-25T20:50:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:30 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]': finished 2024-04-25T20:50:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:30 smithi097 ceph-mon[32667]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T20:50:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:30 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:30 smithi139 ceph-mon[33027]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:30 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/2748120572' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]: dispatch 2024-04-25T20:50:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:30 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]: dispatch 2024-04-25T20:50:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:30 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "683e5552-4261-443b-8552-397edba9e004"}]': finished 2024-04-25T20:50:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:30 smithi139 ceph-mon[33027]: osdmap e36: 6 total, 5 up, 6 in 2024-04-25T20:50:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:30 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:31.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:31 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/1792730251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:31.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:31 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/1792730251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:31.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:31 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/1792730251' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:32 smithi097 ceph-mon[32667]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:32 smithi139 ceph-mon[33027]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:32 smithi096 ceph-mon[29288]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:34 smithi097 ceph-mon[32667]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:34 smithi139 ceph-mon[33027]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:34 smithi096 ceph-mon[29288]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:36.563 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:36 smithi139 ceph-mon[33027]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:36 smithi097 ceph-mon[32667]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:36 smithi096 ceph-mon[29288]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:38.555 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:38 smithi139 ceph-mon[33027]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:38.555 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T20:50:38.555 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:38 smithi097 ceph-mon[32667]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T20:50:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:38 smithi096 ceph-mon[29288]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-25T20:50:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:39.512 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:39 smithi139 ceph-mon[33027]: Deploying daemon osd.5 on smithi139 2024-04-25T20:50:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:39 smithi097 ceph-mon[32667]: Deploying daemon osd.5 on smithi139 2024-04-25T20:50:39.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:39 smithi096 ceph-mon[29288]: Deploying daemon osd.5 on smithi139 2024-04-25T20:50:40.472 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:40 smithi139 ceph-mon[33027]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:40 smithi097 ceph-mon[32667]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:40 smithi096 ceph-mon[29288]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:42.538 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:42 smithi139 ceph-mon[33027]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:42 smithi097 ceph-mon[32667]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:42 smithi096 ceph-mon[29288]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:44.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:44 smithi096 ceph-mon[29288]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:44.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:44 smithi097 ceph-mon[32667]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:44.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:44 smithi139 ceph-mon[33027]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:46.442 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:46 smithi139 ceph-mon[33027]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:46 smithi096 ceph-mon[29288]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:46.528 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 5 on host 'smithi139' 2024-04-25T20:50:46.528 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.526+0000 7f2781ffb700 1 -- 172.21.15.139:0/3912976790 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2784062f40 con 0x7f27740a3c40 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27740a3c40 msgr2=0x7f27740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27740a3c40 0x7f27740a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2778005950 tx=0x7f27780058e0 comp rx=0 tx=0).stop 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 msgr2=0x7f278419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f278419eff0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f277c009610 tx=0x7f277c004ef0 comp rx=0 tx=0).stop 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 shutdown_connections 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27740a3c40 0x7f27740a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:46.531 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2784103540 0x7f278419eff0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:46.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27841019a0 0x7f2784197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:46.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 --2- 172.21.15.139:0/3912976790 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2784102ba0 0x7f2784197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:46.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 >> 172.21.15.139:0/3912976790 conn(0x7f27840fcf40 msgr2=0x7f2784105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:46.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 shutdown_connections 2024-04-25T20:50:46.532 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:50:46.530+0000 7f278af78700 1 -- 172.21.15.139:0/3912976790 wait complete. 2024-04-25T20:50:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:46 smithi097 ceph-mon[32667]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:47.172 DEBUG:teuthology.orchestra.run.smithi139:osd.5> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.5.service 2024-04-25T20:50:47.174 INFO:tasks.cephadm:Deploying osd.6 on smithi096 with /dev/vg_nvme/lv_2... 2024-04-25T20:50:47.175 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T20:50:47.427 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: from='osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:47 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.837 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.837 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.837 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: from='osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:50:47.837 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:50:47.837 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:47.838 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.838 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:47 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: from='osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:47 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:48.731 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-mon[33027]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T20:50:48.731 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-mon[33027]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T20:50:48.731 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:48.731 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-mon[33027]: from='osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:50:48.731 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-mon[33027]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:50:48.732 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T20:50:48.503+0000 7f6729f2a700 -1 osd.5 0 waiting for initial osdmap 2024-04-25T20:50:48.732 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 20:50:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T20:50:48.512+0000 7f6725d1d700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:50:48.830 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:48 smithi096 ceph-mon[29288]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T20:50:48.830 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:48 smithi096 ceph-mon[29288]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T20:50:48.830 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:48 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:48.830 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:48 smithi096 ceph-mon[29288]: from='osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:50:48.830 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:48 smithi096 ceph-mon[29288]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:50:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:48 smithi097 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-25T20:50:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:48 smithi097 ceph-mon[32667]: osdmap e37: 6 total, 5 up, 6 in 2024-04-25T20:50:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:48 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:48 smithi097 ceph-mon[32667]: from='osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:50:48.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:48 smithi097 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:50:49.447 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T20:50:49.447 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T20:50:49.447 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T20:50:49.447 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T20:50:49.447 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112427 s, 933 MB/s 2024-04-25T20:50:49.447 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T20:50:49.795 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:49 smithi139 ceph-mon[33027]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:50:49.795 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:49 smithi139 ceph-mon[33027]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T20:50:49.795 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:49 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:49.795 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:49 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:49.796 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:49 smithi139 ceph-mon[33027]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:49.801 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:49 smithi096 ceph-mon[29288]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:50:49.801 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:49 smithi096 ceph-mon[29288]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T20:50:49.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:49.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:49 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:49.802 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:49 smithi096 ceph-mon[29288]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:49 smithi097 ceph-mon[32667]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:50:49.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:49 smithi097 ceph-mon[32667]: osdmap e38: 6 total, 5 up, 6 in 2024-04-25T20:50:49.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:49 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:49.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:49 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:49.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:49 smithi097 ceph-mon[32667]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-04-25T20:50:50.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-04-25T20:50:50.439 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] boot 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:50.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:50.744 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.764 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] boot 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:50.765 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:50:50.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:50:50.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: osd.5 [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] boot 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: osdmap e39: 6 total, 6 up, 6 in 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: osdmap e40: 6 total, 6 up, 6 in 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:50.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:51.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi139 2024-04-25T20:50:51.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi139 to 8544M 2024-04-25T20:50:51.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:51.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:51.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:51.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:51.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T20:50:51.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:51 smithi096 ceph-mon[29288]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi139 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi139 to 8544M 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T20:50:51.777 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:51 smithi097 ceph-mon[32667]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:50:51.801 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi139 2024-04-25T20:50:51.801 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi139 to 8544M 2024-04-25T20:50:51.801 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:50:51.801 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:51.802 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:50:51.802 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:50:51.802 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: osdmap e41: 6 total, 6 up, 6 in 2024-04-25T20:50:51.802 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:51 smithi139 ceph-mon[33027]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:50:52.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 -- 172.21.15.96:0/3204804431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0103ca0 msgr2=0x7faba0104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 --2- 172.21.15.96:0/3204804431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0103ca0 0x7faba0104100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fab940099c0 tx=0x7fab94009cd0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 -- 172.21.15.96:0/3204804431 shutdown_connections 2024-04-25T20:50:52.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 --2- 172.21.15.96:0/3204804431 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba0104640 0x7faba010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 --2- 172.21.15.96:0/3204804431 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0102aa0 0x7faba0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 --2- 172.21.15.96:0/3204804431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0103ca0 0x7faba0104100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 -- 172.21.15.96:0/3204804431 >> 172.21.15.96:0/3204804431 conn(0x7faba00fe040 msgr2=0x7faba0100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 -- 172.21.15.96:0/3204804431 shutdown_connections 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.090+0000 7faba69f8700 1 -- 172.21.15.96:0/3204804431 wait complete. 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 Processor -- start 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 -- start start 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 0x7faba0198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba0103ca0 0x7faba0199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0104640 0x7faba01a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faba010e510 con 0x7faba0102aa0 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faba010e390 con 0x7faba0103ca0 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba69f8700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faba010e690 con 0x7faba0104640 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7fab9ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 0x7faba0198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7fab9ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 0x7faba0198b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57434/0 (socket says 172.21.15.96:57434) 2024-04-25T20:50:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7fab9ffff700 1 -- 172.21.15.96:0/3512003015 learned_addr learned my addr 172.21.15.96:0/3512003015 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:50:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7fab9f7fe700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba0103ca0 0x7faba0199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.091+0000 7faba4f95700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0104640 0x7faba01a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9ffff700 1 -- 172.21.15.96:0/3512003015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0104640 msgr2=0x7faba01a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9ffff700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0104640 0x7faba01a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9ffff700 1 -- 172.21.15.96:0/3512003015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba0103ca0 msgr2=0x7faba0199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:50:52.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9ffff700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba0103ca0 0x7faba0199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9ffff700 1 -- 172.21.15.96:0/3512003015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba01a07b0 con 0x7faba0102aa0 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9ffff700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 0x7faba0198b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab9000c9c0 tx=0x7fab9000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9000ce60 con 0x7faba0102aa0 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9000f450 con 0x7faba0102aa0 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba01a0aa0 con 0x7faba0102aa0 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab90018660 con 0x7faba0102aa0 2024-04-25T20:50:52.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.092+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba01a0fe0 con 0x7faba0102aa0 2024-04-25T20:50:52.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.093+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faba0069730 con 0x7faba0102aa0 2024-04-25T20:50:52.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.094+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fab9000f920 con 0x7faba0102aa0 2024-04-25T20:50:52.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.094+0000 7fab9d7fa700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab880a3c40 0x7fab880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:50:52.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.095+0000 7fab9f7fe700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab880a3c40 0x7fab880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:50:52.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.095+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4533+0+0 (secure 0 0 0) 0x7fab900cf3d0 con 0x7faba0102aa0 2024-04-25T20:50:52.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.095+0000 7fab9f7fe700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab880a3c40 0x7fab880a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab94009990 tx=0x7fab9400b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:50:52.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.099+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab90014030 con 0x7faba0102aa0 2024-04-25T20:50:52.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:50:52.308+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7faba0062e80 con 0x7fab880a3c40 2024-04-25T20:50:53.167 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:52 smithi096 ceph-mon[29288]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T20:50:53.167 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:52 smithi096 ceph-mon[29288]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:53.167 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:52 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:53.167 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:52 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:53.167 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:52 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:52 smithi097 ceph-mon[32667]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T20:50:53.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:52 smithi097 ceph-mon[32667]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:53.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:52 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:53.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:52 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:53.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:52 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:53.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:52 smithi139 ceph-mon[33027]: osdmap e42: 6 total, 6 up, 6 in 2024-04-25T20:50:53.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:52 smithi139 ceph-mon[33027]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:50:53.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:52 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:50:53.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:52 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:50:53.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:52 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:50:54.184 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:54 smithi096 ceph-mon[29288]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T20:50:54.184 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:54 smithi096 ceph-mon[29288]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 156 KiB/s, 0 objects/s recovering 2024-04-25T20:50:54.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:54 smithi097 ceph-mon[32667]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T20:50:54.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:54 smithi097 ceph-mon[32667]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 156 KiB/s, 0 objects/s recovering 2024-04-25T20:50:54.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:54 smithi139 ceph-mon[33027]: osdmap e43: 6 total, 6 up, 6 in 2024-04-25T20:50:54.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:54 smithi139 ceph-mon[33027]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 156 KiB/s, 0 objects/s recovering 2024-04-25T20:50:55.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:55 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1613790025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a351fac-72bd-41d4-97a0-5f8010ab0eb4"}]: dispatch 2024-04-25T20:50:55.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:55 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1613790025' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a351fac-72bd-41d4-97a0-5f8010ab0eb4"}]': finished 2024-04-25T20:50:55.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:55 smithi097 ceph-mon[32667]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T20:50:55.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:55 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:50:55.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:55 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1613790025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a351fac-72bd-41d4-97a0-5f8010ab0eb4"}]: dispatch 2024-04-25T20:50:55.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:55 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1613790025' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a351fac-72bd-41d4-97a0-5f8010ab0eb4"}]': finished 2024-04-25T20:50:55.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:55 smithi139 ceph-mon[33027]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T20:50:55.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:55 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:50:55.385 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:55 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1613790025' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9a351fac-72bd-41d4-97a0-5f8010ab0eb4"}]: dispatch 2024-04-25T20:50:55.385 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:55 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1613790025' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9a351fac-72bd-41d4-97a0-5f8010ab0eb4"}]': finished 2024-04-25T20:50:55.385 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:55 smithi096 ceph-mon[29288]: osdmap e44: 7 total, 6 up, 7 in 2024-04-25T20:50:55.385 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:55 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:50:56.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:56 smithi097 ceph-mon[32667]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:50:56.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:56 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2339603880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:56.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:56 smithi139 ceph-mon[33027]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:50:56.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:56 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2339603880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:56 smithi096 ceph-mon[29288]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 137 KiB/s, 0 objects/s recovering 2024-04-25T20:50:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:56 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2339603880' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:50:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:50:58 smithi096 ceph-mon[29288]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:50:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:50:58 smithi097 ceph-mon[32667]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:50:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:50:58 smithi139 ceph-mon[33027]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-25T20:51:00.493 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:00 smithi096 ceph-mon[29288]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:51:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:00 smithi097 ceph-mon[32667]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:51:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:00 smithi139 ceph-mon[33027]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:51:02.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:02 smithi096 ceph-mon[29288]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:51:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:02 smithi097 ceph-mon[32667]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:51:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:02 smithi139 ceph-mon[33027]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-04-25T20:51:03.366 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T20:51:03.366 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:03.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T20:51:03.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:03.526 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-25T20:51:03.526 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:04.598 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:04 smithi096 ceph-mon[29288]: Deploying daemon osd.6 on smithi096 2024-04-25T20:51:04.598 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:04 smithi096 ceph-mon[29288]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:04 smithi097 ceph-mon[32667]: Deploying daemon osd.6 on smithi096 2024-04-25T20:51:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:04 smithi097 ceph-mon[32667]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:04 smithi139 ceph-mon[33027]: Deploying daemon osd.6 on smithi096 2024-04-25T20:51:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:04 smithi139 ceph-mon[33027]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:06.482 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:06 smithi096 ceph-mon[29288]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:06 smithi097 ceph-mon[32667]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:06 smithi139 ceph-mon[33027]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:08 smithi097 ceph-mon[32667]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:08 smithi139 ceph-mon[33027]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:08 smithi096 ceph-mon[29288]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:09.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:09.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:09 smithi096 ceph-mon[29288]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:09.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:09.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:09.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:09.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:09 smithi097 ceph-mon[32667]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:09.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:09.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:09.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:09.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:09 smithi139 ceph-mon[33027]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:12.327 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 6 on host 'smithi096' 2024-04-25T20:51:12.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.326+0000 7fab9d7fa700 1 -- 172.21.15.96:0/3512003015 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7faba0062e80 con 0x7fab880a3c40 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab880a3c40 msgr2=0x7fab880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab880a3c40 0x7fab880a6100 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fab94009990 tx=0x7fab9400b870 comp rx=0 tx=0).stop 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 msgr2=0x7faba0198b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 0x7faba0198b50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fab9000c9c0 tx=0x7fab9000ccd0 comp rx=0 tx=0).stop 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 shutdown_connections 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fab880a3c40 0x7fab880a6100 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faba0104640 0x7faba01a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faba0102aa0 0x7faba0198b50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 --2- 172.21.15.96:0/3512003015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faba0103ca0 0x7faba0199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 >> 172.21.15.96:0/3512003015 conn(0x7faba00fe040 msgr2=0x7faba0106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 shutdown_connections 2024-04-25T20:51:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:51:12.329+0000 7faba69f8700 1 -- 172.21.15.96:0/3512003015 wait complete. 2024-04-25T20:51:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:12 smithi097 ceph-mon[32667]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:12 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:12 smithi139 ceph-mon[33027]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:12.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:12.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:12.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:12 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:12 smithi096 ceph-mon[29288]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:12.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:12.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:12.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:12.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:12 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.017 DEBUG:teuthology.orchestra.run.smithi096:osd.6> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.6.service 2024-04-25T20:51:13.020 INFO:tasks.cephadm:Deploying osd.7 on smithi097 with /dev/vg_nvme/lv_2... 2024-04-25T20:51:13.020 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T20:51:13.266 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:51:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:13 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:13 smithi097 ceph-mon[32667]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T20:51:13.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:13 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:13.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:13 smithi139 ceph-mon[33027]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T20:51:13.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:13 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:13.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:13 smithi096 ceph-mon[29288]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-25T20:51:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:14 smithi097 ceph-mon[32667]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:14 smithi097 ceph-mon[32667]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T20:51:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:14 smithi097 ceph-mon[32667]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T20:51:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:14 smithi097 ceph-mon[32667]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:51:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:14.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:14 smithi096 ceph-mon[29288]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:14.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:14 smithi096 ceph-mon[29288]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T20:51:14.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:14 smithi096 ceph-mon[29288]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T20:51:14.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:14 smithi096 ceph-mon[29288]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:51:14.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:14.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:14 smithi139 ceph-mon[33027]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:14.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:14 smithi139 ceph-mon[33027]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-25T20:51:14.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:14 smithi139 ceph-mon[33027]: osdmap e45: 7 total, 6 up, 7 in 2024-04-25T20:51:14.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:14 smithi139 ceph-mon[33027]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:51:14.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T20:51:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T20:51:15.317 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T20:51:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T20:51:15.317 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125744 s, 834 MB/s 2024-04-25T20:51:15.317 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T20:51:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:15 smithi097 ceph-mon[32667]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:51:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:15 smithi097 ceph-mon[32667]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T20:51:15.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:15 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:15 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:15 smithi097 ceph-mon[32667]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' 2024-04-25T20:51:15.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:15 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:15 smithi139 ceph-mon[33027]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:51:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:15 smithi139 ceph-mon[33027]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T20:51:15.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:15 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:15 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:15 smithi139 ceph-mon[33027]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' 2024-04-25T20:51:15.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:15 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.699 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-mon[29288]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:51:15.699 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-mon[29288]: osdmap e46: 7 total, 6 up, 7 in 2024-04-25T20:51:15.699 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.699 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-mon[29288]: from='osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972]' entity='osd.6' 2024-04-25T20:51:15.700 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:15.700 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T20:51:15.251+0000 7fac41192700 -1 osd.6 0 waiting for initial osdmap 2024-04-25T20:51:15.700 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 20:51:15 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T20:51:15.357+0000 7fac38f7d700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:51:15.961 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2024-04-25T20:51:16.208 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.597 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.597 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:16.597 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:16.597 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.597 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:51:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:51:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:17.567 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:17 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi096 2024-04-25T20:51:17.567 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:17 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T20:51:17.568 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:17 smithi097 ceph-mon[32667]: osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] boot 2024-04-25T20:51:17.568 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:17 smithi097 ceph-mon[32667]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T20:51:17.568 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:17 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:17.568 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:17 smithi097 ceph-mon[32667]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:17 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi096 2024-04-25T20:51:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:17 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T20:51:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:17 smithi139 ceph-mon[33027]: osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] boot 2024-04-25T20:51:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:17 smithi139 ceph-mon[33027]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T20:51:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:17 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:17.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:17 smithi139 ceph-mon[33027]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:17 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi096 2024-04-25T20:51:17.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:17 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi096 to 5696M 2024-04-25T20:51:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:17 smithi096 ceph-mon[29288]: osd.6 [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] boot 2024-04-25T20:51:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:17 smithi096 ceph-mon[29288]: osdmap e47: 7 total, 7 up, 7 in 2024-04-25T20:51:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:17 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:51:17.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:17 smithi096 ceph-mon[29288]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 -- 172.21.15.97:0/1843087677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 msgr2=0x7f327c10aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 --2- 172.21.15.97:0/1843087677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c10aef0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f327400b0d0 tx=0x7f327400b3e0 comp rx=0 tx=0).stop 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 -- 172.21.15.97:0/1843087677 shutdown_connections 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 --2- 172.21.15.97:0/1843087677 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c10aef0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 --2- 172.21.15.97:0/1843087677 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c102aa0 0x7f327c102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 --2- 172.21.15.97:0/1843087677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f327c103ca0 0x7f327c104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.775+0000 7f3281676700 1 -- 172.21.15.97:0/1843087677 >> 172.21.15.97:0/1843087677 conn(0x7f327c0fe040 msgr2=0x7f327c100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 -- 172.21.15.97:0/1843087677 shutdown_connections 2024-04-25T20:51:17.779 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 -- 172.21.15.97:0/1843087677 wait complete. 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 Processor -- start 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 -- start start 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f327c102aa0 0x7f327c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c103ca0 0x7f327c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10e540 con 0x7f327c102aa0 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10e3c0 con 0x7f327c104640 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.776+0000 7f3281676700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f327c10e6c0 con 0x7f327c103ca0 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c075370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:36086/0 (socket says 172.21.15.97:36086) 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f327c102aa0 0x7f327c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 -- 172.21.15.97:0/1078186457 learned_addr learned my addr 172.21.15.97:0/1078186457 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T20:51:17.780 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327a7fc700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c103ca0 0x7f327c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 -- 172.21.15.97:0/1078186457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c103ca0 msgr2=0x7f327c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c103ca0 0x7f327c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 -- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f327c102aa0 msgr2=0x7f327c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f327c102aa0 0x7f327c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 -- 172.21.15.97:0/1078186457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c075c20 con 0x7f327c104640 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f327b7fe700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c075370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3274009040 tx=0x7f3274004d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327401f030 con 0x7f327c104640 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c1a2cb0 con 0x7f327c104640 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3274003a00 con 0x7f327c104640 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32740047c0 con 0x7f327c104640 2024-04-25T20:51:17.781 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.777+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1a3100 con 0x7f327c104640 2024-04-25T20:51:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.778+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f327c069730 con 0x7f327c104640 2024-04-25T20:51:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.779+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3274003bb0 con 0x7f327c104640 2024-04-25T20:51:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.780+0000 7f3263fff700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f32640a3c40 0x7f32640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:17.782 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.780+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f327400f020 con 0x7f327c104640 2024-04-25T20:51:17.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.780+0000 7f327affd700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f32640a3c40 0x7f32640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:17.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.781+0000 7f327affd700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f32640a3c40 0x7f32640a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f327c103b00 tx=0x7f326c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:17.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.784+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3274029020 con 0x7f327c104640 2024-04-25T20:51:17.996 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:17.994+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f327c062f40 con 0x7f32640a3c40 2024-04-25T20:51:18.607 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:18 smithi097 ceph-mon[32667]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T20:51:18.607 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:18 smithi097 ceph-mon[32667]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:18.607 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:18.607 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:18.607 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:18.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:18 smithi139 ceph-mon[33027]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T20:51:18.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:18 smithi139 ceph-mon[33027]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:18.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:18.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:18.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:18 smithi096 ceph-mon[29288]: osdmap e48: 7 total, 7 up, 7 in 2024-04-25T20:51:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:18 smithi096 ceph-mon[29288]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:18.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:19.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:19 smithi097 ceph-mon[32667]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:19.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:19 smithi139 ceph-mon[33027]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:19.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:19 smithi096 ceph-mon[29288]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:21.075 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:20 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/1462214773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]: dispatch 2024-04-25T20:51:21.075 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:20 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]: dispatch 2024-04-25T20:51:21.075 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:20 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]': finished 2024-04-25T20:51:21.075 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:20 smithi097 ceph-mon[32667]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T20:51:21.075 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:20 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:21.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:20 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/1462214773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]: dispatch 2024-04-25T20:51:21.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:20 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]: dispatch 2024-04-25T20:51:21.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:20 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]': finished 2024-04-25T20:51:21.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:20 smithi139 ceph-mon[33027]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T20:51:21.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:20 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:20 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/1462214773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]: dispatch 2024-04-25T20:51:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:20 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]: dispatch 2024-04-25T20:51:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:20 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "949efe04-6338-4968-a214-31dec3068888"}]': finished 2024-04-25T20:51:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:20 smithi096 ceph-mon[29288]: osdmap e49: 8 total, 7 up, 8 in 2024-04-25T20:51:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:20 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:21 smithi097 ceph-mon[32667]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:21 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/4252451438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:22.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:21 smithi139 ceph-mon[33027]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:22.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:21 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/4252451438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:22.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:21 smithi096 ceph-mon[29288]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:22.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:21 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/4252451438' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:24.606 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:24 smithi097 ceph-mon[32667]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:24 smithi139 ceph-mon[33027]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:24 smithi096 ceph-mon[29288]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:26 smithi097 ceph-mon[32667]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:26 smithi139 ceph-mon[33027]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:26 smithi096 ceph-mon[29288]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:28 smithi097 ceph-mon[32667]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T20:51:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:28.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:28 smithi139 ceph-mon[33027]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T20:51:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:28 smithi096 ceph-mon[29288]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-25T20:51:28.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:29.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:29 smithi097 ceph-mon[32667]: Deploying daemon osd.7 on smithi097 2024-04-25T20:51:29.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:29 smithi139 ceph-mon[33027]: Deploying daemon osd.7 on smithi097 2024-04-25T20:51:29.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:29 smithi096 ceph-mon[29288]: Deploying daemon osd.7 on smithi097 2024-04-25T20:51:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:30 smithi097 ceph-mon[32667]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:30 smithi139 ceph-mon[33027]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:30.650 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:30 smithi096 ceph-mon[29288]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:32.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:32 smithi097 ceph-mon[32667]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:32 smithi139 ceph-mon[33027]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:32.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:32 smithi096 ceph-mon[29288]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:34.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:34 smithi097 ceph-mon[32667]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:34.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:34.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:34.439 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:34 smithi139 ceph-mon[33027]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:34 smithi096 ceph-mon[29288]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:36.578 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:36 smithi097 ceph-mon[32667]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:36.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:36 smithi139 ceph-mon[33027]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:36 smithi096 ceph-mon[29288]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:37.259 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.259+0000 7f3263fff700 1 -- 172.21.15.97:0/1078186457 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f327c062f40 con 0x7f32640a3c40 2024-04-25T20:51:37.260 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 7 on host 'smithi097' 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f32640a3c40 msgr2=0x7f32640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f32640a3c40 0x7f32640a6100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f327c103b00 tx=0x7f326c00b410 comp rx=0 tx=0).stop 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 msgr2=0x7f327c075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c075370 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3274009040 tx=0x7f3274004d50 comp rx=0 tx=0).stop 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 shutdown_connections 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f32640a3c40 0x7f32640a6100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f327c104640 0x7f327c075370 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:37.262 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f327c102aa0 0x7f327c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:37.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 --2- 172.21.15.97:0/1078186457 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f327c103ca0 0x7f327c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:37.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.260+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 >> 172.21.15.97:0/1078186457 conn(0x7f327c0fe040 msgr2=0x7f327c107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:37.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.261+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 shutdown_connections 2024-04-25T20:51:37.263 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:51:37.261+0000 7f3281676700 1 -- 172.21.15.97:0/1078186457 wait complete. 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:37.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:37.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:37.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:51:37.957 DEBUG:teuthology.orchestra.run.smithi097:osd.7> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.7.service 2024-04-25T20:51:37.959 INFO:tasks.cephadm:Deploying osd.8 on smithi139 with /dev/vg_nvme/lv_2... 2024-04-25T20:51:37.959 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-25T20:51:38.207 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:51:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:38 smithi097 ceph-mon[32667]: from='osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:51:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:38 smithi097 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:51:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:38 smithi139 ceph-mon[33027]: from='osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:51:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:38 smithi139 ceph-mon[33027]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:51:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:38 smithi096 ceph-mon[29288]: from='osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:51:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:38 smithi096 ceph-mon[29288]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-25T20:51:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T20:51:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-mon[32667]: from='osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:51:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-mon[32667]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T20:51:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:51:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-mon[32667]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:39.614 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T20:51:39.432+0000 7fc834e44700 -1 osd.7 0 waiting for initial osdmap 2024-04-25T20:51:39.614 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 20:51:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T20:51:39.442+0000 7fc830c37700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:51:39.714 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:39 smithi139 ceph-mon[33027]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T20:51:39.714 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:39 smithi139 ceph-mon[33027]: from='osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:51:39.714 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:39 smithi139 ceph-mon[33027]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T20:51:39.714 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:39 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:39.715 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:39 smithi139 ceph-mon[33027]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:51:39.715 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:39 smithi139 ceph-mon[33027]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:39.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:39 smithi096 ceph-mon[29288]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-25T20:51:39.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:39 smithi096 ceph-mon[29288]: from='osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:51:39.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:39 smithi096 ceph-mon[29288]: osdmap e50: 8 total, 7 up, 8 in 2024-04-25T20:51:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:39 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:39 smithi096 ceph-mon[29288]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:51:39.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:39 smithi096 ceph-mon[29288]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-25T20:51:40.287 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-25T20:51:40.287 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-25T20:51:40.287 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-04-25T20:51:40.287 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-04-25T20:51:40.287 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126162 s, 831 MB/s 2024-04-25T20:51:40.287 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] boot 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T20:51:40.660 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:40 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.734 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] boot 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T20:51:40.735 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:40 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:51:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:51:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:51:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: osdmap e51: 8 total, 7 up, 8 in 2024-04-25T20:51:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: osd.7 [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] boot 2024-04-25T20:51:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: osdmap e52: 8 total, 8 up, 8 in 2024-04-25T20:51:40.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:40 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:51:40.990 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_2 2024-04-25T20:51:41.239 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:51:42.091 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi097 2024-04-25T20:51:42.091 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.091 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.091 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.091 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.091 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T20:51:42.092 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:41 smithi139 ceph-mon[33027]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:42.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi097 2024-04-25T20:51:42.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T20:51:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:41 smithi097 ceph-mon[32667]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:42.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi097 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi097 to 5696M 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: osdmap e53: 8 total, 8 up, 8 in 2024-04-25T20:51:42.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:41 smithi096 ceph-mon[29288]: pgmap v179: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 -- 172.21.15.139:0/1839884794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 msgr2=0x7f1d24101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 --2- 172.21.15.139:0/1839884794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24101dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1d14009990 tx=0x7f1d14009ca0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 -- 172.21.15.139:0/1839884794 shutdown_connections 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 --2- 172.21.15.139:0/1839884794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d24103540 0x7f1d24109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 --2- 172.21.15.139:0/1839884794 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24101dc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 --2- 172.21.15.139:0/1839884794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24102ba0 0x7f1d24103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 -- 172.21.15.139:0/1839884794 >> 172.21.15.139:0/1839884794 conn(0x7f1d240fcf40 msgr2=0x7f1d240ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:51:42.882 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 -- 172.21.15.139:0/1839884794 shutdown_connections 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.879+0000 7f1d28eee700 1 -- 172.21.15.139:0/1839884794 wait complete. 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 Processor -- start 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 -- start start 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d24102ba0 0x7f1d24199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24103540 0x7f1d241a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410d410 con 0x7f1d24102ba0 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410d290 con 0x7f1d24103540 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d28eee700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2410d590 con 0x7f1d241019a0 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d2259c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d2259c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:52338/0 (socket says 172.21.15.139:52338) 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d2259c700 1 -- 172.21.15.139:0/130785399 learned_addr learned my addr 172.21.15.139:0/130785399 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:51:42.883 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d21d9b700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d24102ba0 0x7f1d24199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.880+0000 7f1d22d9d700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24103540 0x7f1d241a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d2259c700 1 -- 172.21.15.139:0/130785399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24103540 msgr2=0x7f1d241a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d2259c700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24103540 0x7f1d241a01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d2259c700 1 -- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d24102ba0 msgr2=0x7f1d24199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d2259c700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d24102ba0 0x7f1d24199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d2259c700 1 -- 172.21.15.139:0/130785399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d241a0890 con 0x7f1d241019a0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d2259c700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24198c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d14006010 tx=0x7f1d140057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14027070 con 0x7f1d241019a0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d14004080 con 0x7f1d241019a0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d241a0b20 con 0x7f1d241019a0 2024-04-25T20:51:42.884 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d14022dd0 con 0x7f1d241019a0 2024-04-25T20:51:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.881+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d241a0f70 con 0x7f1d241019a0 2024-04-25T20:51:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.882+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d241a1140 con 0x7f1d241019a0 2024-04-25T20:51:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.883+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1d1402a070 con 0x7f1d241019a0 2024-04-25T20:51:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.884+0000 7f1d137fe700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1d0c0a3c40 0x7f1d0c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:51:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.884+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f1d140d0db0 con 0x7f1d241019a0 2024-04-25T20:51:42.885 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.884+0000 7f1d21d9b700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1d0c0a3c40 0x7f1d0c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:51:42.886 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.885+0000 7f1d21d9b700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1d0c0a3c40 0x7f1d0c0a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800ac50 tx=0x7f1d1800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:51:42.888 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:42.888+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d14031020 con 0x7f1d241019a0 2024-04-25T20:51:43.098 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:51:43.097+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f1d240ff180 con 0x7f1d0c0a3c40 2024-04-25T20:51:43.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:43 smithi097 ceph-mon[32667]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T20:51:43.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:43 smithi139 ceph-mon[33027]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T20:51:43.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:43 smithi096 ceph-mon[29288]: osdmap e54: 8 total, 8 up, 8 in 2024-04-25T20:51:44.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T20:51:44.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:44.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:44 smithi097 ceph-mon[32667]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T20:51:44.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T20:51:44.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:44.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:44.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:44.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:44.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:44.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:44 smithi139 ceph-mon[33027]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T20:51:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: osdmap e55: 8 total, 8 up, 8 in 2024-04-25T20:51:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: from='client.24283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:51:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:51:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:51:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:44.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: pgmap v182: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:44.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:44 smithi096 ceph-mon[29288]: osdmap e56: 8 total, 8 up, 8 in 2024-04-25T20:51:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:46 smithi097 ceph-mon[32667]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:46 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/3623314152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]: dispatch 2024-04-25T20:51:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:46 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]: dispatch 2024-04-25T20:51:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:46 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]': finished 2024-04-25T20:51:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:46 smithi097 ceph-mon[32667]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T20:51:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:51:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:46 smithi139 ceph-mon[33027]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:46 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/3623314152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]: dispatch 2024-04-25T20:51:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:46 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]: dispatch 2024-04-25T20:51:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:46 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]': finished 2024-04-25T20:51:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:46 smithi139 ceph-mon[33027]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T20:51:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:51:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:46 smithi096 ceph-mon[29288]: pgmap v184: 1 pgs: 1 remapped; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:46 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/3623314152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]: dispatch 2024-04-25T20:51:46.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:46 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]: dispatch 2024-04-25T20:51:46.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:46 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eb49bb16-6c0d-41a7-8319-cd4e96b9a89d"}]': finished 2024-04-25T20:51:46.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:46 smithi096 ceph-mon[29288]: osdmap e57: 9 total, 8 up, 9 in 2024-04-25T20:51:46.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:51:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:47 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/2631819562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:47 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/2631819562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:47.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:47 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/2631819562' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:51:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:48 smithi097 ceph-mon[32667]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:48 smithi139 ceph-mon[33027]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:48 smithi096 ceph-mon[29288]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:50.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:50 smithi139 ceph-mon[33027]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:50 smithi097 ceph-mon[32667]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:50 smithi096 ceph-mon[29288]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:52.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:52 smithi097 ceph-mon[32667]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:52.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:52 smithi139 ceph-mon[33027]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:52 smithi096 ceph-mon[29288]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:54 smithi097 ceph-mon[32667]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:54 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T20:51:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:54 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:54 smithi139 ceph-mon[33027]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:54 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T20:51:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:54 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:54.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:54 smithi096 ceph-mon[29288]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:54.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:54 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-25T20:51:54.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:54 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:51:55.591 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:55 smithi139 ceph-mon[33027]: Deploying daemon osd.8 on smithi139 2024-04-25T20:51:55.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:55 smithi097 ceph-mon[32667]: Deploying daemon osd.8 on smithi139 2024-04-25T20:51:55.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:55 smithi096 ceph-mon[29288]: Deploying daemon osd.8 on smithi139 2024-04-25T20:51:56.584 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:56 smithi139 ceph-mon[33027]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:56 smithi097 ceph-mon[32667]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:56 smithi096 ceph-mon[29288]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:58 smithi139 ceph-mon[33027]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:58 smithi097 ceph-mon[32667]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:51:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:58 smithi096 ceph-mon[29288]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:00.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:00.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:00.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:00.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:51:59 smithi096 ceph-mon[29288]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:00.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:00.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:00.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:00.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:51:59 smithi097 ceph-mon[32667]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:00.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:51:59 smithi139 ceph-mon[33027]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:02.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:02 smithi139 ceph-mon[33027]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:02.463 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.462+0000 7f1d137fe700 1 -- 172.21.15.139:0/130785399 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1d240ff180 con 0x7f1d0c0a3c40 2024-04-25T20:52:02.463 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 8 on host 'smithi139' 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1d0c0a3c40 msgr2=0x7f1d0c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1d0c0a3c40 0x7f1d0c0a6100 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800ac50 tx=0x7f1d1800a380 comp rx=0 tx=0).stop 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 msgr2=0x7f1d24198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24198c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1d14006010 tx=0x7f1d140057c0 comp rx=0 tx=0).stop 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 shutdown_connections 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1d0c0a3c40 0x7f1d0c0a6100 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1d24103540 0x7f1d241a01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1d241019a0 0x7f1d24198c40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 --2- 172.21.15.139:0/130785399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d24102ba0 0x7f1d24199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.464+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 >> 172.21.15.139:0/130785399 conn(0x7f1d240fcf40 msgr2=0x7f1d240fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.465+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 shutdown_connections 2024-04-25T20:52:02.466 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:02.465+0000 7f1d28eee700 1 -- 172.21.15.139:0/130785399 wait complete. 2024-04-25T20:52:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:02 smithi097 ceph-mon[32667]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:02 smithi096 ceph-mon[29288]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:03.094 DEBUG:teuthology.orchestra.run.smithi139:osd.8> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.8.service 2024-04-25T20:52:03.097 INFO:tasks.cephadm:Deploying osd.9 on smithi096 with /dev/vg_nvme/lv_1... 2024-04-25T20:52:03.097 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T20:52:03.349 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: from='osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:03 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: from='osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:52:03.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:52:03.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:03.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:03 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: from='osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:03.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:03 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:04 smithi139 ceph-mon[33027]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T20:52:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:04 smithi139 ceph-mon[33027]: from='osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:52:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:04 smithi139 ceph-mon[33027]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T20:52:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:04 smithi139 ceph-mon[33027]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:52:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:04 smithi096 ceph-mon[29288]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T20:52:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:04 smithi096 ceph-mon[29288]: from='osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:52:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:04 smithi096 ceph-mon[29288]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T20:52:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:04.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:04 smithi096 ceph-mon[29288]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:52:04.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:04 smithi097 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-25T20:52:04.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:04 smithi097 ceph-mon[32667]: from='osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:52:04.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:04 smithi097 ceph-mon[32667]: osdmap e58: 9 total, 8 up, 9 in 2024-04-25T20:52:04.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:04.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:04 smithi097 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:52:05.382 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T20:52:05.382 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T20:52:05.382 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-04-25T20:52:05.382 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-04-25T20:52:05.382 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125639 s, 835 MB/s 2024-04-25T20:52:05.382 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:05.528 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:05 smithi096 ceph-mon[29288]: from='osd.8 ' entity='osd.8' 2024-04-25T20:52:05.570 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:52:05.570 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T20:52:05.571 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.571 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.571 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T20:52:05.571 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.571 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:05.571 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-mon[33027]: from='osd.8 ' entity='osd.8' 2024-04-25T20:52:05.571 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T20:52:05.390+0000 7f6ab9820700 -1 osd.8 0 waiting for initial osdmap 2024-04-25T20:52:05.571 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 20:52:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T20:52:05.569+0000 7f6ab160b700 -1 osd.8 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:52:05.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: osdmap e59: 9 total, 8 up, 9 in 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: osdmap e60: 9 total, 8 up, 9 in 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-25T20:52:05.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:05 smithi097 ceph-mon[32667]: from='osd.8 ' entity='osd.8' 2024-04-25T20:52:06.085 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-04-25T20:52:06.328 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:52:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:06 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:52:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:06 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:52:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:06 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:06 smithi096 ceph-mon[29288]: osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] boot 2024-04-25T20:52:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:06 smithi096 ceph-mon[29288]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T20:52:06.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:06 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:06.763 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:06 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:52:06.763 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:06 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:52:06.764 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:06 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:06.764 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:06 smithi139 ceph-mon[33027]: osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] boot 2024-04-25T20:52:06.764 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:06 smithi139 ceph-mon[33027]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T20:52:06.764 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:06 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:06 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:52:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:06 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:52:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:06 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:06 smithi097 ceph-mon[32667]: osd.8 [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] boot 2024-04-25T20:52:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:06 smithi097 ceph-mon[32667]: osdmap e61: 9 total, 9 up, 9 in 2024-04-25T20:52:06.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:06 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:52:07.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 -- 172.21.15.96:0/1599991824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430103ca0 msgr2=0x7f4430104100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 --2- 172.21.15.96:0/1599991824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430103ca0 0x7f4430104100 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f44240099c0 tx=0x7f4424009cd0 comp rx=0 tx=0).stop 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 -- 172.21.15.96:0/1599991824 shutdown_connections 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 --2- 172.21.15.96:0/1599991824 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4430104640 0x7f443010aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 --2- 172.21.15.96:0/1599991824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4430102aa0 0x7f4430102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 --2- 172.21.15.96:0/1599991824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430103ca0 0x7f4430104100 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 -- 172.21.15.96:0/1599991824 >> 172.21.15.96:0/1599991824 conn(0x7f44300fe040 msgr2=0x7f4430100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 -- 172.21.15.96:0/1599991824 shutdown_connections 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.962+0000 7f4434f9a700 1 -- 172.21.15.96:0/1599991824 wait complete. 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 Processor -- start 2024-04-25T20:52:07.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 -- start start 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4430102aa0 0x7f4430198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4430103ca0 0x7f44301991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 0x7f44301a0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f443010e5d0 con 0x7f4430104640 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f443010e450 con 0x7f4430103ca0 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f4434f9a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f443010e750 con 0x7f4430102aa0 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 0x7f44301a0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 0x7f44301a0250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33650/0 (socket says 172.21.15.96:33650) 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 -- 172.21.15.96:0/3053228770 learned_addr learned my addr 172.21.15.96:0/3053228770 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442dd9b700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4430103ca0 0x7f44301991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442e59c700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4430102aa0 0x7f4430198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:07.966 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 -- 172.21.15.96:0/3053228770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4430102aa0 msgr2=0x7f4430198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4430102aa0 0x7f4430198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 -- 172.21.15.96:0/3053228770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4430103ca0 msgr2=0x7f44301991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4430103ca0 0x7f44301991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 -- 172.21.15.96:0/3053228770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44301a0920 con 0x7f4430104640 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.963+0000 7f442ed9d700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 0x7f44301a0250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4428007ee0 tx=0x7f442800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.964+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428004bb0 con 0x7f4430104640 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.964+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44301a0c10 con 0x7f4430104640 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.964+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4428004d50 con 0x7f4430104640 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.964+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428005920 con 0x7f4430104640 2024-04-25T20:52:07.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.964+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44301a10d0 con 0x7f4430104640 2024-04-25T20:52:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.965+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4430069730 con 0x7f4430104640 2024-04-25T20:52:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.966+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f442801a020 con 0x7f4430104640 2024-04-25T20:52:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.966+0000 7f441f7fe700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44180a3c40 0x7f44180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.966+0000 7f442e59c700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44180a3c40 0x7f44180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.966+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f44280cf120 con 0x7f4430104640 2024-04-25T20:52:07.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.967+0000 7f442e59c700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44180a3c40 0x7f44180a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4430199b80 tx=0x7f4420009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:07.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:07.971+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4428010070 con 0x7f4430104640 2024-04-25T20:52:08.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi139 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi139 to 5696M 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:07 smithi097 ceph-mon[32667]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi139 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi139 to 5696M 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:08.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.124 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:07 smithi139 ceph-mon[33027]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:08.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:08.185+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f4430062e80 con 0x7f44180a3c40 2024-04-25T20:52:08.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi139 2024-04-25T20:52:08.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.186 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi139 to 5696M 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: osdmap e62: 9 total, 9 up, 9 in 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:08.187 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:07 smithi096 ceph-mon[29288]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:09.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:09 smithi096 ceph-mon[29288]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T20:52:09.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:09 smithi096 ceph-mon[29288]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:09.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:09.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:09.121 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:09.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:09 smithi097 ceph-mon[32667]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T20:52:09.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:09 smithi097 ceph-mon[32667]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:09.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:09.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:09.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:09.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:09 smithi139 ceph-mon[33027]: osdmap e63: 9 total, 9 up, 9 in 2024-04-25T20:52:09.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:09 smithi139 ceph-mon[33027]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:09.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:09.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:09.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:10.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:10 smithi097 ceph-mon[32667]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T20:52:10.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:10 smithi097 ceph-mon[32667]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:10.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:10 smithi139 ceph-mon[33027]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T20:52:10.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:10 smithi139 ceph-mon[33027]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:10 smithi096 ceph-mon[29288]: osdmap e64: 9 total, 9 up, 9 in 2024-04-25T20:52:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:10 smithi096 ceph-mon[29288]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:11 smithi097 ceph-mon[32667]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T20:52:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:11 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3970627915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffaf81c6-340a-416a-8c8d-ed179d853403"}]: dispatch 2024-04-25T20:52:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:11 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3970627915' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffaf81c6-340a-416a-8c8d-ed179d853403"}]': finished 2024-04-25T20:52:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:11 smithi097 ceph-mon[32667]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T20:52:11.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:11.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:11 smithi139 ceph-mon[33027]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T20:52:11.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:11 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3970627915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffaf81c6-340a-416a-8c8d-ed179d853403"}]: dispatch 2024-04-25T20:52:11.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:11 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3970627915' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffaf81c6-340a-416a-8c8d-ed179d853403"}]': finished 2024-04-25T20:52:11.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:11 smithi139 ceph-mon[33027]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T20:52:11.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:11 smithi096 ceph-mon[29288]: osdmap e65: 9 total, 9 up, 9 in 2024-04-25T20:52:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:11 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3970627915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ffaf81c6-340a-416a-8c8d-ed179d853403"}]: dispatch 2024-04-25T20:52:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:11 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3970627915' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ffaf81c6-340a-416a-8c8d-ed179d853403"}]': finished 2024-04-25T20:52:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:11 smithi096 ceph-mon[29288]: osdmap e66: 10 total, 9 up, 10 in 2024-04-25T20:52:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:12.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:12 smithi097 ceph-mon[32667]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:52:12.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:12 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2181756096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:12.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:12 smithi139 ceph-mon[33027]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:52:12.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:12 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2181756096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:12 smithi096 ceph-mon[29288]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:52:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:12 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2181756096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:14 smithi097 ceph-mon[32667]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T20:52:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:14 smithi139 ceph-mon[33027]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T20:52:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:14 smithi096 ceph-mon[29288]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-04-25T20:52:16.545 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:16 smithi096 ceph-mon[29288]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T20:52:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:16 smithi097 ceph-mon[32667]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T20:52:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:16 smithi139 ceph-mon[33027]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-04-25T20:52:18.462 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:18 smithi096 ceph-mon[29288]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T20:52:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:18 smithi097 ceph-mon[32667]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T20:52:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:18 smithi139 ceph-mon[33027]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-25T20:52:19.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:19 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T20:52:19.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:19 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:19.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:19 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T20:52:19.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:19 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:19.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:19 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-25T20:52:19.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:19 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:20 smithi097 ceph-mon[32667]: Deploying daemon osd.9 on smithi096 2024-04-25T20:52:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:20 smithi097 ceph-mon[32667]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T20:52:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:20 smithi139 ceph-mon[33027]: Deploying daemon osd.9 on smithi096 2024-04-25T20:52:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:20 smithi139 ceph-mon[33027]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T20:52:20.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:20 smithi096 ceph-mon[29288]: Deploying daemon osd.9 on smithi096 2024-04-25T20:52:20.687 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:20 smithi096 ceph-mon[29288]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-04-25T20:52:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:22 smithi096 ceph-mon[29288]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:22 smithi097 ceph-mon[32667]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:22.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:22 smithi139 ceph-mon[33027]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:24 smithi097 ceph-mon[32667]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:24 smithi139 ceph-mon[33027]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:24 smithi096 ceph-mon[29288]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:25.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:25 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:25.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:25 smithi096 ceph-mon[29288]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:25.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:25.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:25.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:25 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:25.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:25 smithi097 ceph-mon[32667]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:25.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:25.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:25.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:25 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:25.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:25 smithi139 ceph-mon[33027]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:28.496 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.495+0000 7f441f7fe700 1 -- 172.21.15.96:0/3053228770 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4430062e80 con 0x7f44180a3c40 2024-04-25T20:52:28.496 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 9 on host 'smithi096' 2024-04-25T20:52:28.496 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:28 smithi096 ceph-mon[29288]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:28.496 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.497 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.497 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:28.497 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:28.497 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:28 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44180a3c40 msgr2=0x7f44180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44180a3c40 0x7f44180a6100 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4430199b80 tx=0x7f4420009190 comp rx=0 tx=0).stop 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 msgr2=0x7f44301a0250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 0x7f44301a0250 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4428007ee0 tx=0x7f442800d3b0 comp rx=0 tx=0).stop 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 shutdown_connections 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44180a3c40 0x7f44180a6100 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4430104640 0x7f44301a0250 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4430102aa0 0x7f4430198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 --2- 172.21.15.96:0/3053228770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4430103ca0 0x7f44301991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 >> 172.21.15.96:0/3053228770 conn(0x7f44300fe040 msgr2=0x7f4430106fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 shutdown_connections 2024-04-25T20:52:28.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:52:28.497+0000 7f4434f9a700 1 -- 172.21.15.96:0/3053228770 wait complete. 2024-04-25T20:52:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:28 smithi097 ceph-mon[32667]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:28.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:28.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:28 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:28 smithi139 ceph-mon[33027]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:28 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.155 DEBUG:teuthology.orchestra.run.smithi096:osd.9> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.9.service 2024-04-25T20:52:29.158 INFO:tasks.cephadm:Deploying osd.10 on smithi097 with /dev/vg_nvme/lv_1... 2024-04-25T20:52:29.158 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T20:52:29.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:29 smithi096 ceph-mon[29288]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T20:52:29.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.380 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:29.407 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:52:29.528 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:29 smithi097 ceph-mon[32667]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T20:52:29.528 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.528 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.529 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:29.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:29 smithi139 ceph-mon[33027]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-25T20:52:29.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:29.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:30 smithi097 ceph-mon[32667]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:30 smithi097 ceph-mon[32667]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T20:52:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:30 smithi097 ceph-mon[32667]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T20:52:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:30 smithi097 ceph-mon[32667]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:52:30.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:30 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:30 smithi139 ceph-mon[33027]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:30 smithi139 ceph-mon[33027]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T20:52:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:30 smithi139 ceph-mon[33027]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T20:52:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:30 smithi139 ceph-mon[33027]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:52:30.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:30 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:30.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:30 smithi096 ceph-mon[29288]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:30.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:30 smithi096 ceph-mon[29288]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-25T20:52:30.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:30 smithi096 ceph-mon[29288]: osdmap e67: 10 total, 9 up, 10 in 2024-04-25T20:52:30.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:30 smithi096 ceph-mon[29288]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-04-25T20:52:30.646 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:30 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T20:52:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T20:52:31.398 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2024-04-25T20:52:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2024-04-25T20:52:31.398 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.015281 s, 686 MB/s 2024-04-25T20:52:31.398 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2024-04-25T20:52:31.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-mon[29288]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:52:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-mon[29288]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T20:52:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-mon[29288]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' 2024-04-25T20:52:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.500 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T20:52:31.212+0000 7f0aed2ef700 -1 osd.9 0 waiting for initial osdmap 2024-04-25T20:52:31.500 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 20:52:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T20:52:31.295+0000 7f0ae50da700 -1 osd.9 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:52:31.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:31 smithi097 ceph-mon[32667]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:52:31.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:31 smithi097 ceph-mon[32667]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T20:52:31.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:31 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:31 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:31 smithi097 ceph-mon[32667]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' 2024-04-25T20:52:31.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:31 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:31 smithi139 ceph-mon[33027]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-04-25T20:52:31.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:31 smithi139 ceph-mon[33027]: osdmap e68: 10 total, 9 up, 10 in 2024-04-25T20:52:31.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:31 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:31 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:31.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:31 smithi139 ceph-mon[33027]: from='osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151]' entity='osd.9' 2024-04-25T20:52:31.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:31 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:32.001 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2024-04-25T20:52:32.238 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T20:52:32.607 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:32 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:52:32.608 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:32 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:52:32.608 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:32 smithi097 ceph-mon[32667]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:32.608 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:32 smithi097 ceph-mon[32667]: osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] boot 2024-04-25T20:52:32.608 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:32 smithi097 ceph-mon[32667]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T20:52:32.608 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:32 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:52:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:32 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:52:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:32 smithi139 ceph-mon[33027]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:32 smithi139 ceph-mon[33027]: osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] boot 2024-04-25T20:52:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:32 smithi139 ceph-mon[33027]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T20:52:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:32.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:32 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:52:32.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:32 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:52:32.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:32 smithi096 ceph-mon[29288]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-25T20:52:32.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:32 smithi096 ceph-mon[29288]: osd.9 [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] boot 2024-04-25T20:52:32.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:32 smithi096 ceph-mon[29288]: osdmap e69: 10 total, 10 up, 10 in 2024-04-25T20:52:32.743 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi096 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:33.563 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:33.564 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi096 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi096 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi096 to 4272M 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:33.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 -- 172.21.15.97:0/3664002674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 msgr2=0x7f52601011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 --2- 172.21.15.97:0/3664002674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601011d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5250009990 tx=0x7f5250009ca0 comp rx=0 tx=0).stop 2024-04-25T20:52:33.783 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 -- 172.21.15.97:0/3664002674 shutdown_connections 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 --2- 172.21.15.97:0/3664002674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260104730 0x7f526010afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 --2- 172.21.15.97:0/3664002674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601011d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 --2- 172.21.15.97:0/3664002674 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5260103d90 0x7f52601041f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 -- 172.21.15.97:0/3664002674 >> 172.21.15.97:0/3664002674 conn(0x7f52600fc710 msgr2=0x7f52600feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 -- 172.21.15.97:0/3664002674 shutdown_connections 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 -- 172.21.15.97:0/3664002674 wait complete. 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 Processor -- start 2024-04-25T20:52:33.784 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.782+0000 7f52664b6700 1 -- start start 2024-04-25T20:52:33.785 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260103d90 0x7f5260101d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5260104730 0x7f5260102280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f526010e5d0 con 0x7f5260103d90 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f526010e450 con 0x7f5260100db0 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f526010e750 con 0x7f5260104730 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:53058/0 (socket says 172.21.15.97:53058) 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 -- 172.21.15.97:0/3601659639 learned_addr learned my addr 172.21.15.97:0/3601659639 (peer_addr_for_me v2:172.21.15.97:0/0) 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525f7fe700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260103d90 0x7f5260101d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 -- 172.21.15.97:0/3601659639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5260104730 msgr2=0x7f5260102280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5260104730 0x7f5260102280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:33.786 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 -- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260103d90 msgr2=0x7f5260101d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260103d90 0x7f5260101d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 -- 172.21.15.97:0/3601659639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5260102b30 con 0x7f5260100db0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525f7fe700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260103d90 0x7f5260101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525ffff700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601036f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f525000b5c0 tx=0x7f5250005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250027070 con 0x7f5260100db0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52500040c0 con 0x7f5260100db0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52601a2d70 con 0x7f5260100db0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5250022dc0 con 0x7f5260100db0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.783+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52601a31c0 con 0x7f5260100db0 2024-04-25T20:52:33.787 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.784+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f526002df80 con 0x7f5260100db0 2024-04-25T20:52:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.785+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f525002a070 con 0x7f5260100db0 2024-04-25T20:52:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.786+0000 7f525d7fa700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f52480a3c40 0x7f52480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.786+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6261+0+0 (secure 0 0 0) 0x7f52500d12c0 con 0x7f5260100db0 2024-04-25T20:52:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.786+0000 7f525f7fe700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f52480a3c40 0x7f52480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:33.788 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.787+0000 7f525f7fe700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f52480a3c40 0x7f52480a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5254007900 tx=0x7f5254008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:33.791 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.790+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5250031070 con 0x7f5260100db0 2024-04-25T20:52:33.998 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:33.997+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5260062e80 con 0x7f52480a3c40 2024-04-25T20:52:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:34 smithi097 ceph-mon[32667]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:34 smithi097 ceph-mon[32667]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T20:52:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:34 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:34 smithi139 ceph-mon[33027]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:34 smithi139 ceph-mon[33027]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T20:52:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:34.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:34 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:34 smithi096 ceph-mon[29288]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:34 smithi096 ceph-mon[29288]: osdmap e70: 10 total, 10 up, 10 in 2024-04-25T20:52:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:34.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:34 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:35.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:35 smithi097 ceph-mon[32667]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:35.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:35 smithi139 ceph-mon[33027]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:35.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:35 smithi096 ceph-mon[29288]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:52:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:36 smithi097 ceph-mon[32667]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:36 smithi139 ceph-mon[33027]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:36 smithi096 ceph-mon[29288]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:37 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/3632090195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]: dispatch 2024-04-25T20:52:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:37 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]: dispatch 2024-04-25T20:52:37.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:37 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]': finished 2024-04-25T20:52:37.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:37 smithi097 ceph-mon[32667]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T20:52:37.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:37 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:37.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:37 smithi097 ceph-mon[32667]: from='client.? 172.21.15.97:0/3849755200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:37.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:37 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/3632090195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]: dispatch 2024-04-25T20:52:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:37 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]: dispatch 2024-04-25T20:52:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:37 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]': finished 2024-04-25T20:52:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:37 smithi139 ceph-mon[33027]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T20:52:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:37 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:37.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:37 smithi139 ceph-mon[33027]: from='client.? 172.21.15.97:0/3849755200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:37.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:37 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/3632090195' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]: dispatch 2024-04-25T20:52:37.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:37 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]: dispatch 2024-04-25T20:52:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:37 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "872e6405-0b55-4088-ab33-a8a3f4b7ac3e"}]': finished 2024-04-25T20:52:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:37 smithi096 ceph-mon[29288]: osdmap e71: 11 total, 10 up, 11 in 2024-04-25T20:52:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:37 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:37.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:37 smithi096 ceph-mon[29288]: from='client.? 172.21.15.97:0/3849755200' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:52:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:38 smithi097 ceph-mon[32667]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:38 smithi139 ceph-mon[33027]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:38 smithi096 ceph-mon[29288]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:39 smithi097 ceph-mon[32667]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:39.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:39 smithi139 ceph-mon[33027]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:39.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:39 smithi096 ceph-mon[29288]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:42 smithi096 ceph-mon[29288]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:42 smithi097 ceph-mon[32667]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:42 smithi139 ceph-mon[33027]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:44 smithi097 ceph-mon[32667]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T20:52:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:44 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:44 smithi139 ceph-mon[33027]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T20:52:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:44 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:44 smithi096 ceph-mon[29288]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-25T20:52:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:44 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:45.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:45 smithi097 ceph-mon[32667]: Deploying daemon osd.10 on smithi097 2024-04-25T20:52:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:45 smithi139 ceph-mon[33027]: Deploying daemon osd.10 on smithi097 2024-04-25T20:52:45.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:45 smithi096 ceph-mon[29288]: Deploying daemon osd.10 on smithi097 2024-04-25T20:52:46.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:46 smithi097 ceph-mon[32667]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:46 smithi139 ceph-mon[33027]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:46 smithi096 ceph-mon[29288]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:48 smithi097 ceph-mon[32667]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:48 smithi139 ceph-mon[33027]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:48 smithi096 ceph-mon[29288]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:50.333 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:50 smithi097 ceph-mon[32667]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:50.333 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:50.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:50.334 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:50 smithi139 ceph-mon[33027]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:50 smithi096 ceph-mon[29288]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:52 smithi097 ceph-mon[32667]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:52 smithi139 ceph-mon[33027]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:52 smithi096 ceph-mon[29288]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:53.313 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.312+0000 7f525d7fa700 1 -- 172.21.15.97:0/3601659639 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f5260062e80 con 0x7f52480a3c40 2024-04-25T20:52:53.313 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 10 on host 'smithi097' 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f52480a3c40 msgr2=0x7f52480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f52480a3c40 0x7f52480a6100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5254007900 tx=0x7f5254008040 comp rx=0 tx=0).stop 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 msgr2=0x7f52601036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601036f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f525000b5c0 tx=0x7f5250005b00 comp rx=0 tx=0).stop 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 shutdown_connections 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f52480a3c40 0x7f52480a6100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5260104730 0x7f5260102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:53.315 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5260100db0 0x7f52601036f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 --2- 172.21.15.97:0/3601659639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5260103d90 0x7f5260101d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 >> 172.21.15.97:0/3601659639 conn(0x7f52600fc710 msgr2=0x7f52601070b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 shutdown_connections 2024-04-25T20:52:53.316 INFO:teuthology.orchestra.run.smithi097.stderr:2024-04-25T20:52:53.315+0000 7f52664b6700 1 -- 172.21.15.97:0/3601659639 wait complete. 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:53.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:53.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.981 DEBUG:teuthology.orchestra.run.smithi097:osd.10> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.10.service 2024-04-25T20:52:53.983 INFO:tasks.cephadm:Deploying osd.11 on smithi139 with /dev/vg_nvme/lv_1... 2024-04-25T20:52:53.984 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-25T20:52:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:52:54.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:54.230 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: from='osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: from='osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:52:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:54 smithi139 ceph-mon[33027]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:52:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: from='osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:52:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:52:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T20:52:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T20:52:54.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:54.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: from='osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:52:54.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:54 smithi097 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:52:54.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: from='osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:52:54.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-25T20:52:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-25T20:52:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: osdmap e72: 11 total, 10 up, 11 in 2024-04-25T20:52:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: from='osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:52:55.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:54 smithi096 ceph-mon[29288]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2024-04-25T20:52:56.330 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-25T20:52:56.330 INFO:teuthology.orchestra.run.smithi139.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-25T20:52:56.330 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10+0 records in 2024-04-25T20:52:56.330 INFO:teuthology.orchestra.run.smithi139.stderr:10+0 records out 2024-04-25T20:52:56.330 INFO:teuthology.orchestra.run.smithi139.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0123923 s, 846 MB/s 2024-04-25T20:52:56.330 INFO:teuthology.orchestra.run.smithi139.stderr:--> Zapping successful for: 2024-04-25T20:52:56.415 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:56 smithi139 ceph-mon[33027]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:56.415 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:56 smithi139 ceph-mon[33027]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:52:56.415 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:56 smithi139 ceph-mon[33027]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T20:52:56.415 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:56.415 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:56.415 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:56 smithi139 ceph-mon[33027]: from='osd.10 ' entity='osd.10' 2024-04-25T20:52:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-mon[32667]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-mon[32667]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:52:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-mon[32667]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T20:52:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-mon[32667]: from='osd.10 ' entity='osd.10' 2024-04-25T20:52:56.614 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T20:52:56.228+0000 7f1eb5bcd700 -1 osd.10 0 waiting for initial osdmap 2024-04-25T20:52:56.614 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 20:52:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T20:52:56.344+0000 7f1ead9b8700 -1 osd.10 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:52:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:56 smithi096 ceph-mon[29288]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:56 smithi096 ceph-mon[29288]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2024-04-25T20:52:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:56 smithi096 ceph-mon[29288]: osdmap e73: 11 total, 10 up, 11 in 2024-04-25T20:52:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:56.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:56 smithi096 ceph-mon[29288]: from='osd.10 ' entity='osd.10' 2024-04-25T20:52:57.092 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi139:vg_nvme/lv_1 2024-04-25T20:52:57.338 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:57.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:57.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:57 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:52:57.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:52:57.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:57.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T20:52:57.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:57.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:57 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: osdmap e74: 11 total, 10 up, 11 in 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:57.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:52:57.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:57 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:52:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi097 2024-04-25T20:52:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T20:52:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] boot 2024-04-25T20:52:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T20:52:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi097 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] boot 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T20:52:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi097 2024-04-25T20:52:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi097 to 4272M 2024-04-25T20:52:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-25T20:52:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: osd.10 [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] boot 2024-04-25T20:52:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: osdmap e75: 11 total, 11 up, 11 in 2024-04-25T20:52:58.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.834+0000 7fdd933e3700 1 -- 172.21.15.139:0/3290909053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 msgr2=0x7fdd8c102770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.834+0000 7fdd933e3700 1 --2- 172.21.15.139:0/3290909053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c102770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdd80009990 tx=0x7fdd80009ca0 comp rx=0 tx=0).stop 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- 172.21.15.139:0/3290909053 shutdown_connections 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 --2- 172.21.15.139:0/3290909053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd8c102cb0 0x7fdd8c109560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 --2- 172.21.15.139:0/3290909053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd8c101110 0x7fdd8c101530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 --2- 172.21.15.139:0/3290909053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c102770 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- 172.21.15.139:0/3290909053 >> 172.21.15.139:0/3290909053 conn(0x7fdd8c0fc6d0 msgr2=0x7fdd8c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- 172.21.15.139:0/3290909053 shutdown_connections 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- 172.21.15.139:0/3290909053 wait complete. 2024-04-25T20:52:58.850 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 Processor -- start 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- start start 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd8c101110 0x7fdd8c10ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd8c102cb0 0x7fdd8c19ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10d4a0 con 0x7fdd8c101110 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10d320 con 0x7fdd8c102cb0 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd933e3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c10d620 con 0x7fdd8c102310 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.835+0000 7fdd9097e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:56298/0 (socket says 172.21.15.139:56298) 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 -- 172.21.15.139:0/4235990635 learned_addr learned my addr 172.21.15.139:0/4235990635 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9117f700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd8c101110 0x7fdd8c10ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 -- 172.21.15.139:0/4235990635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd8c102cb0 msgr2=0x7fdd8c19ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd8c102cb0 0x7fdd8c19ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:58.851 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 -- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd8c101110 msgr2=0x7fdd8c10ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd8c101110 0x7fdd8c10ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 -- 172.21.15.139:0/4235990635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c1a08a0 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd9097e700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c19d940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd800047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80027070 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd80004100 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c1a0b30 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80022d70 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.836+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c1a0fe0 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.837+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c069730 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.838+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdd8002a070 con 0x7fdd8c102310 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.839+0000 7fdd8a7fc700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdd740a3cc0 0x7fdd740a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:52:58.852 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.839+0000 7fdd9117f700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdd740a3cc0 0x7fdd740a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:52:58.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.839+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 6723+0+0 (secure 0 0 0) 0x7fdd800d0e50 con 0x7fdd8c102310 2024-04-25T20:52:58.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.840+0000 7fdd9117f700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdd740a3cc0 0x7fdd740a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c102170 tx=0x7fdd8400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:52:58.853 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:58.843+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd80031080 con 0x7fdd8c102310 2024-04-25T20:52:59.050 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:52:59.049+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdd8c062e80 con 0x7fdd740a3cc0 2024-04-25T20:52:59.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:59 smithi097 ceph-mon[32667]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T20:52:59.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:59.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:59.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:52:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:59.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:59 smithi139 ceph-mon[33027]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T20:52:59.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:59.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:52:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:52:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:59 smithi096 ceph-mon[29288]: osdmap e76: 11 total, 11 up, 11 in 2024-04-25T20:52:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-25T20:52:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-25T20:52:59.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:52:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:00 smithi097 ceph-mon[32667]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:00.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:00 smithi097 ceph-mon[32667]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:00.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:00 smithi097 ceph-mon[32667]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T20:53:00.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:00 smithi097 ceph-mon[32667]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T20:53:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:00 smithi139 ceph-mon[33027]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:00 smithi139 ceph-mon[33027]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:00 smithi139 ceph-mon[33027]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T20:53:00.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:00 smithi139 ceph-mon[33027]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T20:53:00.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:00 smithi096 ceph-mon[29288]: from='client.24340 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi139:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:00.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:00 smithi096 ceph-mon[29288]: pgmap v241: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:00.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:00 smithi096 ceph-mon[29288]: osdmap e77: 11 total, 11 up, 11 in 2024-04-25T20:53:00.651 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:00 smithi096 ceph-mon[29288]: osdmap e78: 11 total, 11 up, 11 in 2024-04-25T20:53:02.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T20:53:02.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-04-25T20:53:02.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/2898845932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]: dispatch 2024-04-25T20:53:02.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]: dispatch 2024-04-25T20:53:02.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]': finished 2024-04-25T20:53:02.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T20:53:02.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:02 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:02.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T20:53:02.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-04-25T20:53:02.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/2898845932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]: dispatch 2024-04-25T20:53:02.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]: dispatch 2024-04-25T20:53:02.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]': finished 2024-04-25T20:53:02.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T20:53:02.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:02 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:02.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: osdmap e79: 11 total, 11 up, 11 in 2024-04-25T20:53:02.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2024-04-25T20:53:02.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/2898845932' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]: dispatch 2024-04-25T20:53:02.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]: dispatch 2024-04-25T20:53:02.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a27fcf8-65a4-491e-9f62-8e597415e90c"}]': finished 2024-04-25T20:53:02.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: osdmap e80: 12 total, 11 up, 12 in 2024-04-25T20:53:02.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:02 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:03.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:03 smithi097 ceph-mon[32667]: from='client.? 172.21.15.139:0/1042164779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:03.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:03 smithi139 ceph-mon[33027]: from='client.? 172.21.15.139:0/1042164779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:03.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:03 smithi096 ceph-mon[29288]: from='client.? 172.21.15.139:0/1042164779' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-25T20:53:04.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:04 smithi097 ceph-mon[32667]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:04.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:04 smithi139 ceph-mon[33027]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:04 smithi096 ceph-mon[29288]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-25T20:53:06.591 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:06 smithi139 ceph-mon[33027]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T20:53:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:06 smithi097 ceph-mon[32667]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T20:53:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:06 smithi096 ceph-mon[29288]: pgmap v248: 1 pgs: 1 peering; 577 KiB data, 334 MiB used, 983 GiB / 983 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-25T20:53:08.602 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:08 smithi139 ceph-mon[33027]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:53:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:08 smithi097 ceph-mon[32667]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:53:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:08 smithi096 ceph-mon[29288]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-04-25T20:53:10.591 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:10 smithi139 ceph-mon[33027]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:10.591 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T20:53:10.591 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:10 smithi097 ceph-mon[32667]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T20:53:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:10 smithi096 ceph-mon[29288]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-25T20:53:10.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:11.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:11 smithi097 ceph-mon[32667]: Deploying daemon osd.11 on smithi139 2024-04-25T20:53:11.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:11 smithi139 ceph-mon[33027]: Deploying daemon osd.11 on smithi139 2024-04-25T20:53:11.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:11 smithi096 ceph-mon[29288]: Deploying daemon osd.11 on smithi139 2024-04-25T20:53:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:12 smithi097 ceph-mon[32667]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:12 smithi139 ceph-mon[33027]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:12 smithi096 ceph-mon[29288]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:14.551 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:14 smithi139 ceph-mon[33027]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:14 smithi097 ceph-mon[32667]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:14 smithi096 ceph-mon[29288]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:16.516 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:16 smithi139 ceph-mon[33027]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:16.517 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:16.517 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:16.517 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:16 smithi097 ceph-mon[32667]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:16 smithi096 ceph-mon[29288]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:18.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:18 smithi139 ceph-mon[33027]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:18 smithi097 ceph-mon[32667]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:18 smithi096 ceph-mon[29288]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:18.918 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.918+0000 7fdd8a7fc700 1 -- 172.21.15.139:0/4235990635 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdd8c062e80 con 0x7fdd740a3cc0 2024-04-25T20:53:18.919 INFO:teuthology.orchestra.run.smithi139.stdout:Created osd(s) 11 on host 'smithi139' 2024-04-25T20:53:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdd740a3cc0 msgr2=0x7fdd740a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdd740a3cc0 0x7fdd740a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c102170 tx=0x7fdd8400d040 comp rx=0 tx=0).stop 2024-04-25T20:53:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 msgr2=0x7fdd8c19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c19d940 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdd80006010 tx=0x7fdd800047b0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 shutdown_connections 2024-04-25T20:53:18.921 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdd740a3cc0 0x7fdd740a6180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdd8c102cb0 0x7fdd8c19ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd8c101110 0x7fdd8c10ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 --2- 172.21.15.139:0/4235990635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdd8c102310 0x7fdd8c19d940 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 >> 172.21.15.139:0/4235990635 conn(0x7fdd8c0fc6d0 msgr2=0x7fdd8c1056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 shutdown_connections 2024-04-25T20:53:18.922 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:18.920+0000 7fdd933e3700 1 -- 172.21.15.139:0/4235990635 wait complete. 2024-04-25T20:53:19.826 DEBUG:teuthology.orchestra.run.smithi139:osd.11> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.11.service 2024-04-25T20:53:19.829 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-25T20:53:19.829 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T20:53:20.097 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:20.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:19 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:20.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:19 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:20.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:19 smithi139 ceph-mon[33027]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:20.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:19 smithi139 ceph-mon[33027]: from='osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:53:20.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:19 smithi139 ceph-mon[33027]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:53:20.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:19 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:20.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:19 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:20.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:19 smithi096 ceph-mon[29288]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:20.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:19 smithi096 ceph-mon[29288]: from='osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:53:20.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:19 smithi096 ceph-mon[29288]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:53:20.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:19 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:20.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:19 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:20.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:19 smithi097 ceph-mon[32667]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:20.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:19 smithi097 ceph-mon[32667]: from='osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:53:20.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:19 smithi097 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-25T20:53:21.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T20:53:21.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T20:53:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: from='osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:53:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:53:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:21.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:20 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:21.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T20:53:21.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T20:53:21.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:21.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: from='osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:53:21.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:53:21.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:21.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:20 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:21.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-25T20:53:21.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: osdmap e81: 12 total, 11 up, 12 in 2024-04-25T20:53:21.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:21.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: from='osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:53:21.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]: dispatch 2024-04-25T20:53:21.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:21.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:20 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:21.873 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T20:53:21.838+0000 7f4816f7f700 -1 osd.11 0 waiting for initial osdmap 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 -- 172.21.15.96:0/3502671058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 msgr2=0x7fb8d81041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 --2- 172.21.15.96:0/3502671058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d81041f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8009990 tx=0x7fb8c8009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 -- 172.21.15.96:0/3502671058 shutdown_connections 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 --2- 172.21.15.96:0/3502671058 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8d8104730 0x7fb8d810afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 --2- 172.21.15.96:0/3502671058 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d8100db0 0x7fb8d81011d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 --2- 172.21.15.96:0/3502671058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d81041f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 -- 172.21.15.96:0/3502671058 >> 172.21.15.96:0/3502671058 conn(0x7fb8d80fc710 msgr2=0x7fb8d80feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 -- 172.21.15.96:0/3502671058 shutdown_connections 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 -- 172.21.15.96:0/3502671058 wait complete. 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 Processor -- start 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.026+0000 7fb8e0286700 1 -- start start 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8d8100db0 0x7fb8d8198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d8199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d8104730 0x7fb8d81a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810e550 con 0x7fb8d8103d90 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810e3d0 con 0x7fb8d8104730 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb8d810e6d0 con 0x7fb8d8100db0 2024-04-25T20:53:22.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d8199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:22.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d8199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41296/0 (socket says 172.21.15.96:41296) 2024-04-25T20:53:22.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 -- 172.21.15.96:0/147500939 learned_addr learned my addr 172.21.15.96:0/147500939 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:22.031 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8de022700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8d8100db0 0x7fb8d8198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:22.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8de823700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d8104730 0x7fb8d81a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:22.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 -- 172.21.15.96:0/147500939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8d8100db0 msgr2=0x7fb8d8198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:22.033 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8d8100db0 0x7fb8d8198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 -- 172.21.15.96:0/147500939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d8104730 msgr2=0x7fb8d81a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d8104730 0x7fb8d81a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 -- 172.21.15.96:0/147500939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8d81a0820 con 0x7fb8d8103d90 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8dd821700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d8199100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80036f0 tx=0x7fb8c80057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8027070 con 0x7fb8d8103d90 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8c8004000 con 0x7fb8d8103d90 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8d81a0ab0 con 0x7fb8d8103d90 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8022c50 con 0x7fb8d8103d90 2024-04-25T20:53:22.034 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.027+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8d81a0f60 con 0x7fb8d8103d90 2024-04-25T20:53:22.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.028+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8d8069730 con 0x7fb8d8103d90 2024-04-25T20:53:22.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:21 smithi096 ceph-mon[29288]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:53:22.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:21 smithi096 ceph-mon[29288]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T20:53:22.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:22.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:22.035 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:21 smithi096 ceph-mon[29288]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:22.036 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:21 smithi096 ceph-mon[29288]: from='osd.11 ' entity='osd.11' 2024-04-25T20:53:22.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.030+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb8c802a070 con 0x7fb8d8103d90 2024-04-25T20:53:22.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.031+0000 7fb8cf7fe700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb8c40a3cc0 0x7fb8c40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:22.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.031+0000 7fb8de022700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb8c40a3cc0 0x7fb8c40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:22.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.031+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 6847+0+0 (secure 0 0 0) 0x7fb8c80d14f0 con 0x7fb8d8103d90 2024-04-25T20:53:22.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.031+0000 7fb8de022700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb8c40a3cc0 0x7fb8c40a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8d806af70 tx=0x7fb8d4007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:22.037 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.034+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8c8031020 con 0x7fb8d8103d90 2024-04-25T20:53:22.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.233+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb8d8063650 con 0x7fb8d8103d90 2024-04-25T20:53:22.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.233+0000 7fb8cf7fe700 1 -- 172.21.15.96:0/147500939 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v82) ==== 74+0+133 (secure 0 0 0) 0x7fb8c8013070 con 0x7fb8d8103d90 2024-04-25T20:53:22.235 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:22.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb8c40a3cc0 msgr2=0x7fb8c40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:22.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb8c40a3cc0 0x7fb8c40a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb8d806af70 tx=0x7fb8d4007400 comp rx=0 tx=0).stop 2024-04-25T20:53:22.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 msgr2=0x7fb8d8199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:22.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d8199100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80036f0 tx=0x7fb8c80057c0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.237 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 shutdown_connections 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb8c40a3cc0 0x7fb8c40a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb8d8104730 0x7fb8d81a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb8d8100db0 0x7fb8d8198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 --2- 172.21.15.96:0/147500939 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8d8103d90 0x7fb8d8199100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 >> 172.21.15.96:0/147500939 conn(0x7fb8d80fc710 msgr2=0x7fb8d8107110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 shutdown_connections 2024-04-25T20:53:22.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:22.236+0000 7fb8e0286700 1 -- 172.21.15.96:0/147500939 wait complete. 2024-04-25T20:53:22.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:21 smithi097 ceph-mon[32667]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:53:22.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:21 smithi097 ceph-mon[32667]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T20:53:22.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:22.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:22.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:21 smithi097 ceph-mon[32667]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:22.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:21 smithi097 ceph-mon[32667]: from='osd.11 ' entity='osd.11' 2024-04-25T20:53:22.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 20:53:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T20:53:22.016+0000 7f480e569700 -1 osd.11 82 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T20:53:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-mon[33027]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi139", "root=default"]}]': finished 2024-04-25T20:53:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-mon[33027]: osdmap e82: 12 total, 11 up, 12 in 2024-04-25T20:53:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-mon[33027]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:22.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:21 smithi139 ceph-mon[33027]: from='osd.11 ' entity='osd.11' 2024-04-25T20:53:22.858 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":82,"num_osds":12,"num_up_osds":11,"osd_up_since":1714078377,"num_in_osds":12,"osd_in_since":1714078381,"num_remapped_pgs":0} 2024-04-25T20:53:23.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:22 smithi096 ceph-mon[29288]: purged_snaps scrub starts 2024-04-25T20:53:23.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:22 smithi096 ceph-mon[29288]: purged_snaps scrub ok 2024-04-25T20:53:23.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:23.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:22 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/147500939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:53:23.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:22 smithi097 ceph-mon[32667]: purged_snaps scrub starts 2024-04-25T20:53:23.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:22 smithi097 ceph-mon[32667]: purged_snaps scrub ok 2024-04-25T20:53:23.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:23.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:22 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/147500939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:53:23.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:22 smithi139 ceph-mon[33027]: purged_snaps scrub starts 2024-04-25T20:53:23.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:22 smithi139 ceph-mon[33027]: purged_snaps scrub ok 2024-04-25T20:53:23.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:23.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:22 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/147500939' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:53:23.860 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-25T20:53:24.096 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:24.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:24.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] boot 2024-04-25T20:53:24.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T20:53:24.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:24.218 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:24.219 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:23 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] boot 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:24.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:24.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:24.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:23 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: osd.11 [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] boot 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: osdmap e83: 12 total, 12 up, 12 in 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 338 MiB used, 983 GiB / 983 GiB avail 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:53:24.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:24.374 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:23 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:25.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:24 smithi096 ceph-mon[29288]: Detected new or changed devices on smithi139 2024-04-25T20:53:25.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:24 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi139 to 4272M 2024-04-25T20:53:25.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:24 smithi096 ceph-mon[29288]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T20:53:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:24 smithi097 ceph-mon[32667]: Detected new or changed devices on smithi139 2024-04-25T20:53:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:24 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi139 to 4272M 2024-04-25T20:53:25.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:24 smithi097 ceph-mon[32667]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T20:53:25.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:24 smithi139 ceph-mon[33027]: Detected new or changed devices on smithi139 2024-04-25T20:53:25.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:24 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi139 to 4272M 2024-04-25T20:53:25.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:24 smithi139 ceph-mon[33027]: osdmap e84: 12 total, 12 up, 12 in 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.819+0000 7f85536ed700 1 -- 172.21.15.96:0/2279452689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 msgr2=0x7f854c103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.819+0000 7f85536ed700 1 --2- 172.21.15.96:0/2279452689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c103000 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f853c0099c0 tx=0x7f853c009cd0 comp rx=0 tx=0).stop 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- 172.21.15.96:0/2279452689 shutdown_connections 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 --2- 172.21.15.96:0/2279452689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c103540 0x7f854c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 --2- 172.21.15.96:0/2279452689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c1019a0 0x7f854c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 --2- 172.21.15.96:0/2279452689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c103000 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:25.823 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- 172.21.15.96:0/2279452689 >> 172.21.15.96:0/2279452689 conn(0x7f854c0fcf40 msgr2=0x7f854c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- 172.21.15.96:0/2279452689 shutdown_connections 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- 172.21.15.96:0/2279452689 wait complete. 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 Processor -- start 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- start start 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c1019a0 0x7f854c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c1991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c103540 0x7f854c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10d490 con 0x7f854c102ba0 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10d310 con 0x7f854c103540 2024-04-25T20:53:25.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f85536ed700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f854c10d610 con 0x7f854c1019a0 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.820+0000 7f8550c88700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c1991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c1991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41324/0 (socket says 172.21.15.96:41324) 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 -- 172.21.15.96:0/4271278903 learned_addr learned my addr 172.21.15.96:0/4271278903 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8551489700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c1019a0 0x7f854c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 -- 172.21.15.96:0/4271278903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c1019a0 msgr2=0x7f854c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c1019a0 0x7f854c198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 -- 172.21.15.96:0/4271278903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c103540 msgr2=0x7f854c1a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8551c8a700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c103540 0x7f854c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c103540 0x7f854c1a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 -- 172.21.15.96:0/4271278903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854c1a0930 con 0x7f854c102ba0 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8551489700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c1019a0 0x7f854c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8551c8a700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c103540 0x7f854c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:53:25.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f8550c88700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c1991f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f853c00b580 tx=0x7f853c003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:25.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c027070 con 0x7f854c102ba0 2024-04-25T20:53:25.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f853c022470 con 0x7f854c102ba0 2024-04-25T20:53:25.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853c00f6e0 con 0x7f854c102ba0 2024-04-25T20:53:25.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854c1a0bc0 con 0x7f854c102ba0 2024-04-25T20:53:25.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.821+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f854c1a10a0 con 0x7f854c102ba0 2024-04-25T20:53:25.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.822+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854c069730 con 0x7f854c102ba0 2024-04-25T20:53:25.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.823+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f853c02a070 con 0x7f854c102ba0 2024-04-25T20:53:25.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.824+0000 7f85427fc700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f85380a3c40 0x7f85380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:25.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.824+0000 7f8551489700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f85380a3c40 0x7f85380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:25.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.824+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f853c0d0e20 con 0x7f854c102ba0 2024-04-25T20:53:25.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.824+0000 7f8551489700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f85380a3c40 0x7f85380a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8548005fd0 tx=0x7f8548005de0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:25.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:25.829+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f853c013070 con 0x7f854c102ba0 2024-04-25T20:53:26.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.026+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f854c1a14d0 con 0x7f854c102ba0 2024-04-25T20:53:26.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.026+0000 7f85427fc700 1 -- 172.21.15.96:0/4271278903 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v84) ==== 74+0+133 (secure 0 0 0) 0x7f853c09dd30 con 0x7f854c102ba0 2024-04-25T20:53:26.027 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:26.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f85380a3c40 msgr2=0x7f85380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:26.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f85380a3c40 0x7f85380a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8548005fd0 tx=0x7f8548005de0 comp rx=0 tx=0).stop 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 msgr2=0x7f854c1991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c1991f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f853c00b580 tx=0x7f853c003960 comp rx=0 tx=0).stop 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 shutdown_connections 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f85380a3c40 0x7f85380a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f854c103540 0x7f854c1a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f854c1019a0 0x7f854c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 --2- 172.21.15.96:0/4271278903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f854c102ba0 0x7f854c1991f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 >> 172.21.15.96:0/4271278903 conn(0x7f854c0fcf40 msgr2=0x7f854c06a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 shutdown_connections 2024-04-25T20:53:26.030 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:26.028+0000 7f85536ed700 1 -- 172.21.15.96:0/4271278903 wait complete. 2024-04-25T20:53:26.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:25 smithi096 ceph-mon[29288]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:25 smithi097 ceph-mon[32667]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:26.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:25 smithi139 ceph-mon[33027]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:26.667 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":84,"num_osds":12,"num_up_osds":12,"osd_up_since":1714078402,"num_in_osds":12,"osd_in_since":1714078381,"num_remapped_pgs":0} 2024-04-25T20:53:26.667 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:53:26.903 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:27.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4271278903' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:53:27.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:26 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4271278903' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:53:27.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:26 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4271278903' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-25T20:53:27.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:27 smithi096 ceph-mon[29288]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:28.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:27 smithi097 ceph-mon[32667]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:28.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:27 smithi139 ceph-mon[33027]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 -- 172.21.15.96:0/2248693247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42001034c0 msgr2=0x7f420010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2248693247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42001034c0 0x7f420010f8e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f41f0009990 tx=0x7f41f0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 -- 172.21.15.96:0/2248693247 shutdown_connections 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2248693247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f42001034c0 0x7f420010f8e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2248693247 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200108a10 0x7f4200108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2248693247 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4200102b00 0x7f4200102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 -- 172.21.15.96:0/2248693247 >> 172.21.15.96:0/2248693247 conn(0x7f4200079c00 msgr2=0x7f420007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 -- 172.21.15.96:0/2248693247 shutdown_connections 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.565+0000 7f4206fb1700 1 -- 172.21.15.96:0/2248693247 wait complete. 2024-04-25T20:53:28.568 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 Processor -- start 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 -- start start 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200102b00 0x7f4200074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001034c0 0x7f4200075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 0x7f42000757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112c70 con 0x7f4200108a10 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112af0 con 0x7f42001034c0 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f4206fb1700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4200112df0 con 0x7f4200102b00 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f420554e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 0x7f42000757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f420554e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 0x7f42000757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38102/0 (socket says 172.21.15.96:38102) 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f420554e700 1 -- 172.21.15.96:0/2374197564 learned_addr learned my addr 172.21.15.96:0/2374197564 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.566+0000 7f41fffff700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001034c0 0x7f4200075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f4204d4d700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200102b00 0x7f4200074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f420554e700 1 -- 172.21.15.96:0/2374197564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200102b00 msgr2=0x7f4200074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:28.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f420554e700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200102b00 0x7f4200074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f420554e700 1 -- 172.21.15.96:0/2374197564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001034c0 msgr2=0x7f4200075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f420554e700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001034c0 0x7f4200075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f420554e700 1 -- 172.21.15.96:0/2374197564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42001a7590 con 0x7f4200108a10 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f41fffff700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001034c0 0x7f4200075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f4204d4d700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200102b00 0x7f4200074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f420554e700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 0x7f42000757c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005910 tx=0x7f41f00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0027070 con 0x7f4200108a10 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f0004020 con 0x7f4200108a10 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42001a7820 con 0x7f4200108a10 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0022d40 con 0x7f4200108a10 2024-04-25T20:53:28.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.567+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42001a7cd0 con 0x7f4200108a10 2024-04-25T20:53:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.568+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4200069730 con 0x7f4200108a10 2024-04-25T20:53:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.569+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f41f002a070 con 0x7f4200108a10 2024-04-25T20:53:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.570+0000 7f41fdffb700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.570+0000 7f4204d4d700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.570+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f41f00d1570 con 0x7f4200108a10 2024-04-25T20:53:28.571 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.570+0000 7f4204d4d700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009fa0 tx=0x7f41f8009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:28.573 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.574+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41f0031070 con 0x7f4200108a10 2024-04-25T20:53:28.775 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.775+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42001a7ff0 con 0x7f4200108a10 2024-04-25T20:53:28.776 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.776+0000 7f41fdffb700 1 -- 172.21.15.96:0/2374197564 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+18421 (secure 0 0 0) 0x7f41f0013070 con 0x7f4200108a10 2024-04-25T20:53:28.776 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:28.777 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":84,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:53:24.429822+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.778+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f41ec0a3cc0 msgr2=0x7f41ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.778+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f41f8009fa0 tx=0x7f41f8009570 comp rx=0 tx=0).stop 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.778+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 msgr2=0x7f42000757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.778+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 0x7f42000757c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005910 tx=0x7f41f00047b0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 shutdown_connections 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f41ec0a3cc0 0x7f41ec0a6180 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4200108a10 0x7f42000757c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4200102b00 0x7f4200074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 --2- 172.21.15.96:0/2374197564 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42001034c0 0x7f4200075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 >> 172.21.15.96:0/2374197564 conn(0x7f4200079c00 msgr2=0x7f4200100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:28.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 shutdown_connections 2024-04-25T20:53:28.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:28.779+0000 7f4206fb1700 1 -- 172.21.15.96:0/2374197564 wait complete. 2024-04-25T20:53:29.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:28 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2374197564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:53:29.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:28 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2374197564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:53:29.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:28 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2374197564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:53:29.502 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:49:37.227783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T20:53:29.502 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-25T20:53:29.742 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:30.220 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:29 smithi096 ceph-mon[29288]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:29 smithi097 ceph-mon[32667]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:30.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:29 smithi139 ceph-mon[33027]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:31.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 -- 172.21.15.96:0/1354217140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44107990 msgr2=0x7f9e44107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:31.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 --2- 172.21.15.96:0/1354217140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44107990 0x7f9e44107d70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009990 tx=0x7f9e34009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 -- 172.21.15.96:0/1354217140 shutdown_connections 2024-04-25T20:53:31.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 --2- 172.21.15.96:0/1354217140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e44102300 0x7f9e4410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 --2- 172.21.15.96:0/1354217140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44107990 0x7f9e44107d70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 --2- 172.21.15.96:0/1354217140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44101940 0x7f9e44101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 -- 172.21.15.96:0/1354217140 >> 172.21.15.96:0/1354217140 conn(0x7f9e440788f0 msgr2=0x7f9e44078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:31.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 -- 172.21.15.96:0/1354217140 shutdown_connections 2024-04-25T20:53:31.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.477+0000 7f9e4898f700 1 -- 172.21.15.96:0/1354217140 wait complete. 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 Processor -- start 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 -- start start 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 0x7f9e44199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e44102300 0x7f9e4419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44107990 0x7f9e441a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111ae0 con 0x7f9e44101940 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111960 con 0x7f9e44102300 2024-04-25T20:53:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4898f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111c60 con 0x7f9e44107990 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 0x7f9e44199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 0x7f9e44199f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38118/0 (socket says 172.21.15.96:38118) 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e4259c700 1 -- 172.21.15.96:0/3417266076 learned_addr learned my addr 172.21.15.96:0/3417266076 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e42d9d700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44107990 0x7f9e441a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.478+0000 7f9e39d9b700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e44102300 0x7f9e4419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4259c700 1 -- 172.21.15.96:0/3417266076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44107990 msgr2=0x7f9e441a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4259c700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44107990 0x7f9e441a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4259c700 1 -- 172.21.15.96:0/3417266076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e44102300 msgr2=0x7f9e4419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4259c700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e44102300 0x7f9e4419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4259c700 1 -- 172.21.15.96:0/3417266076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e44197d70 con 0x7f9e44101940 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e42d9d700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44107990 0x7f9e441a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4259c700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 0x7f9e44199f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e340057c0 tx=0x7f9e340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34027070 con 0x7f9e44101940 2024-04-25T20:53:31.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e34003f20 con 0x7f9e44101940 2024-04-25T20:53:31.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e34022b30 con 0x7f9e44101940 2024-04-25T20:53:31.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e44198000 con 0x7f9e44101940 2024-04-25T20:53:31.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.479+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e441984e0 con 0x7f9e44101940 2024-04-25T20:53:31.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.480+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e44069730 con 0x7f9e44101940 2024-04-25T20:53:31.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.482+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9e3402a070 con 0x7f9e44101940 2024-04-25T20:53:31.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.482+0000 7f9e3bfff700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9e300a3c40 0x7f9e300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:31.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.483+0000 7f9e39d9b700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9e300a3c40 0x7f9e300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:31.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.483+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f9e340d1360 con 0x7f9e44101940 2024-04-25T20:53:31.484 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.483+0000 7f9e39d9b700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9e300a3c40 0x7f9e300a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9e44078ff0 tx=0x7f9e2c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:31.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.486+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9e34031020 con 0x7f9e44101940 2024-04-25T20:53:31.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.687+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9e4402df80 con 0x7f9e44101940 2024-04-25T20:53:31.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.687+0000 7f9e3bfff700 1 -- 172.21.15.96:0/3417266076 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f9e34013070 con 0x7f9e44101940 2024-04-25T20:53:31.688 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T20:53:31.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9e300a3c40 msgr2=0x7f9e300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:31.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9e300a3c40 0x7f9e300a6100 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9e44078ff0 tx=0x7f9e2c0073d0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 msgr2=0x7f9e44199f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 0x7f9e44199f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9e340057c0 tx=0x7f9e340057f0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 shutdown_connections 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9e300a3c40 0x7f9e300a6100 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e44107990 0x7f9e441a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e44101940 0x7f9e44199f80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 --2- 172.21.15.96:0/3417266076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e44102300 0x7f9e4419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 >> 172.21.15.96:0/3417266076 conn(0x7f9e440788f0 msgr2=0x7f9e44100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 shutdown_connections 2024-04-25T20:53:31.691 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:31.689+0000 7f9e4898f700 1 -- 172.21.15.96:0/3417266076 wait complete. 2024-04-25T20:53:32.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:32 smithi097 ceph-mon[32667]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:32.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:32 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3417266076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T20:53:32.398 INFO:tasks.cephadm:Adding mds.a on smithi096 2024-04-25T20:53:32.398 INFO:tasks.cephadm:Adding mds.d on smithi096 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.g on smithi096 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.j on smithi096 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.b on smithi097 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.e on smithi097 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.h on smithi097 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.k on smithi097 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.c on smithi139 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.f on smithi139 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.i on smithi139 2024-04-25T20:53:32.399 INFO:tasks.cephadm:Adding mds.l on smithi139 2024-04-25T20:53:32.399 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l' 2024-04-25T20:53:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:32 smithi139 ceph-mon[33027]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:32.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:32 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3417266076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T20:53:32.650 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T20:53:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:32 smithi096 ceph-mon[29288]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:32 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3417266076' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 -- 172.21.15.139:0/3268226052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac100e20 msgr2=0x7fdfac101240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/3268226052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac100e20 0x7fdfac101240 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c009990 tx=0x7fdf9c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 -- 172.21.15.139:0/3268226052 shutdown_connections 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/3268226052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac1047a0 0x7fdfac10b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/3268226052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac100e20 0x7fdfac101240 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/3268226052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac103e00 0x7fdfac104260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 -- 172.21.15.139:0/3268226052 >> 172.21.15.139:0/3268226052 conn(0x7fdfac0fc780 msgr2=0x7fdfac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 -- 172.21.15.139:0/3268226052 shutdown_connections 2024-04-25T20:53:34.222 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.218+0000 7fdfb10b0700 1 -- 172.21.15.139:0/3268226052 wait complete. 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.219+0000 7fdfb10b0700 1 Processor -- start 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.219+0000 7fdfb10b0700 1 -- start start 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac100e20 0x7fdfac198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103e00 0x7fdfac1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 0x7fdfac1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10e5c0 con 0x7fdfac103e00 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10e440 con 0x7fdfac100e20 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10e740 con 0x7fdfac1047a0 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 0x7fdfac1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 0x7fdfac1a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.139:39116/0 (socket says 172.21.15.139:39116) 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 -- 172.21.15.139:0/1109069601 learned_addr learned my addr 172.21.15.139:0/1109069601 (peer_addr_for_me v2:172.21.15.139:0/0) 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfaa59c700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103e00 0x7fdfac1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfaad9d700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac100e20 0x7fdfac198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 -- 172.21.15.139:0/1109069601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac100e20 msgr2=0x7fdfac198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:34.223 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac100e20 0x7fdfac198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 -- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103e00 msgr2=0x7fdfac1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103e00 0x7fdfac1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 -- 172.21.15.139:0/1109069601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac1a08b0 con 0x7fdfac1047a0 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfab59e700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 0x7fdfac1a01e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4007ee0 tx=0x7fdfa400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa400dbd0 con 0x7fdfac1047a0 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac1a0ba0 con 0x7fdfac1047a0 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfa400dd70 con 0x7fdfac1047a0 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.220+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfa4005990 con 0x7fdfac1047a0 2024-04-25T20:53:34.224 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.221+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac1a1090 con 0x7fdfac1047a0 2024-04-25T20:53:34.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.221+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac069730 con 0x7fdfac1047a0 2024-04-25T20:53:34.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.223+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdfa401a020 con 0x7fdfac1047a0 2024-04-25T20:53:34.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.224+0000 7fdf93fff700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdf940a3c40 0x7fdf940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:34.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.224+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fdfa40cfdb0 con 0x7fdfac1047a0 2024-04-25T20:53:34.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.224+0000 7fdfaad9d700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdf940a3c40 0x7fdf940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:34.225 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.224+0000 7fdfaad9d700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdf940a3c40 0x7fdf940a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0038a0 tx=0x7fdf9c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:34.228 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.228+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdfa4010070 con 0x7fdfac1047a0 2024-04-25T20:53:34.437 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.436+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l", "target": ["mon-mgr", ""]}) -- 0x7fdfac063140 con 0x7fdf940a3c40 2024-04-25T20:53:34.445 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.444+0000 7fdf93fff700 1 -- 172.21.15.139:0/1109069601 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fdfac063140 con 0x7fdf940a3c40 2024-04-25T20:53:34.445 INFO:teuthology.orchestra.run.smithi139.stdout:Scheduled mds.all update... 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.446+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdf940a3c40 msgr2=0x7fdf940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.446+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdf940a3c40 0x7fdf940a6100 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c0038a0 tx=0x7fdf9c005fd0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.446+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 msgr2=0x7fdfac1a01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.446+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 0x7fdfac1a01e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdfa4007ee0 tx=0x7fdfa400d3b0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 shutdown_connections 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fdf940a3c40 0x7fdf940a6100 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfac1047a0 0x7fdfac1a01e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.448 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfac100e20 0x7fdfac198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 --2- 172.21.15.139:0/1109069601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfac103e00 0x7fdfac1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 >> 172.21.15.139:0/1109069601 conn(0x7fdfac0fc780 msgr2=0x7fdfac1070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 shutdown_connections 2024-04-25T20:53:34.449 INFO:teuthology.orchestra.run.smithi139.stderr:2024-04-25T20:53:34.447+0000 7fdfb10b0700 1 -- 172.21.15.139:0/1109069601 wait complete. 2024-04-25T20:53:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:34 smithi097 ceph-mon[32667]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:34 smithi139 ceph-mon[33027]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:34 smithi096 ceph-mon[29288]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:35.063 DEBUG:teuthology.orchestra.run.smithi096:mds.a> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.a.service 2024-04-25T20:53:35.066 DEBUG:teuthology.orchestra.run.smithi096:mds.d> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.d.service 2024-04-25T20:53:35.110 DEBUG:teuthology.orchestra.run.smithi096:mds.g> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.g.service 2024-04-25T20:53:35.154 DEBUG:teuthology.orchestra.run.smithi096:mds.j> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.j.service 2024-04-25T20:53:35.197 DEBUG:teuthology.orchestra.run.smithi097:mds.b> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.b.service 2024-04-25T20:53:35.199 DEBUG:teuthology.orchestra.run.smithi097:mds.e> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.e.service 2024-04-25T20:53:35.242 DEBUG:teuthology.orchestra.run.smithi097:mds.h> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.h.service 2024-04-25T20:53:35.285 DEBUG:teuthology.orchestra.run.smithi097:mds.k> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.k.service 2024-04-25T20:53:35.328 DEBUG:teuthology.orchestra.run.smithi139:mds.c> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.c.service 2024-04-25T20:53:35.330 DEBUG:teuthology.orchestra.run.smithi139:mds.f> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.f.service 2024-04-25T20:53:35.373 DEBUG:teuthology.orchestra.run.smithi139:mds.i> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.i.service 2024-04-25T20:53:35.416 DEBUG:teuthology.orchestra.run.smithi139:mds.l> sudo journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.l.service 2024-04-25T20:53:35.460 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-25T20:53:35.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:53:35.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:35.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l;count:12 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: Deploying daemon mds.l on smithi139 2024-04-25T20:53:35.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:35 smithi097 ceph-mon[32667]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l;count:12 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: Deploying daemon mds.l on smithi139 2024-04-25T20:53:35.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:35 smithi139 ceph-mon[33027]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:35.694 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: Saving service mds.all spec with placement smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l;count:12 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:35.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: Deploying daemon mds.l on smithi139 2024-04-25T20:53:35.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:35 smithi096 ceph-mon[29288]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:37.373 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 systemd[1]: Starting Ceph mds.l for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:53:37.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 -- 172.21.15.96:0/448354833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028107920 msgr2=0x7fb028107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 --2- 172.21.15.96:0/448354833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028107920 0x7fb028107d00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb018009990 tx=0x7fb018009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 -- 172.21.15.96:0/448354833 shutdown_connections 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 --2- 172.21.15.96:0/448354833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028102330 0x7fb02810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 --2- 172.21.15.96:0/448354833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028107920 0x7fb028107d00 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 --2- 172.21.15.96:0/448354833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb028101970 0x7fb028101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 -- 172.21.15.96:0/448354833 >> 172.21.15.96:0/448354833 conn(0x7fb0280788f0 msgr2=0x7fb028078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 -- 172.21.15.96:0/448354833 shutdown_connections 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.397+0000 7fb02e71b700 1 -- 172.21.15.96:0/448354833 wait complete. 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 Processor -- start 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 -- start start 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028101970 0x7fb028199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 0x7fb028199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:37.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb028107920 0x7fb02819db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111bb0 con 0x7fb028102330 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111a30 con 0x7fb028101970 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02e71b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb028111d30 con 0x7fb028107920 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb0277fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 0x7fb028199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb0277fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 0x7fb028199770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48424/0 (socket says 172.21.15.96:48424) 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb0277fe700 1 -- 172.21.15.96:0/3384324409 learned_addr learned my addr 172.21.15.96:0/3384324409 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.398+0000 7fb02ccb8700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb028107920 0x7fb02819db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb027fff700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028101970 0x7fb028199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0277fe700 1 -- 172.21.15.96:0/3384324409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb028107920 msgr2=0x7fb02819db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0277fe700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb028107920 0x7fb02819db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0277fe700 1 -- 172.21.15.96:0/3384324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028101970 msgr2=0x7fb028199230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0277fe700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028101970 0x7fb028199230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0277fe700 1 -- 172.21.15.96:0/3384324409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02819e240 con 0x7fb028102330 2024-04-25T20:53:37.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb027fff700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028101970 0x7fb028199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0277fe700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 0x7fb028199770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d980 tx=0x7fb01c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c017070 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02819e530 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0281a5db0 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb01c005020 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.399+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01c009b50 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.400+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb028069730 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.401+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb01c009cf0 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.402+0000 7fb0257fa700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0100a3c40 0x7fb0100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.402+0000 7fb027fff700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0100a3c40 0x7fb0100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.402+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb01c0cfe70 con 0x7fb028102330 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.402+0000 7fb027fff700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0100a3c40 0x7fb0100a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b5c0 tx=0x7fb018005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:37.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.406+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb01c020020 con 0x7fb028102330 2024-04-25T20:53:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.648+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb02819a590 con 0x7fb028102330 2024-04-25T20:53:37.649 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.648+0000 7fb0257fa700 1 -- 172.21.15.96:0/3384324409 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb01c09ccf0 con 0x7fb028102330 2024-04-25T20:53:37.650 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:37.650 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.650+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0100a3c40 msgr2=0x7fb0100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.650+0000 7fb02e71b700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0100a3c40 0x7fb0100a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b5c0 tx=0x7fb018005fb0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 msgr2=0x7fb028199770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 0x7fb028199770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fb01c00d980 tx=0x7fb01c00dc90 comp rx=0 tx=0).stop 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 shutdown_connections 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0100a3c40 0x7fb0100a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb028107920 0x7fb02819db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028101970 0x7fb028199230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 --2- 172.21.15.96:0/3384324409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb028102330 0x7fb028199770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 >> 172.21.15.96:0/3384324409 conn(0x7fb0280788f0 msgr2=0x7fb02806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 shutdown_connections 2024-04-25T20:53:37.652 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:37.651+0000 7fb02e71b700 1 -- 172.21.15.96:0/3384324409 wait complete. 2024-04-25T20:53:37.653 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 1 2024-04-25T20:53:37.772 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 podman[62656]: 2024-04-25 20:53:37.382622835 +0000 UTC m=+0.017674612 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:53:37.772 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 podman[62656]: 2024-04-25 20:53:37.519753161 +0000 UTC m=+0.154804929 container create 226c852ad8e88ded88c1b9cc68f90828b3dc8f6434a364c54e40a6cd3b594453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T20:53:37.772 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 podman[62656]: 2024-04-25 20:53:37.772049306 +0000 UTC m=+0.407101073 container init 226c852ad8e88ded88c1b9cc68f90828b3dc8f6434a364c54e40a6cd3b594453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T20:53:38.123 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 podman[62656]: 2024-04-25 20:53:37.774757589 +0000 UTC m=+0.409809385 container start 226c852ad8e88ded88c1b9cc68f90828b3dc8f6434a364c54e40a6cd3b594453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T20:53:38.123 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 ceph-mds[62675]: -- 172.21.15.139:0/522591982 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563085497a40 con 0x56308546a800 2024-04-25T20:53:38.123 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l[62671]: starting mds.l at 2024-04-25T20:53:38.123 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 bash[62656]: 226c852ad8e88ded88c1b9cc68f90828b3dc8f6434a364c54e40a6cd3b594453 2024-04-25T20:53:38.123 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 20:53:37 smithi139 systemd[1]: Started Ceph mds.l for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3384324409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6832/3174744460,v1:172.21.15.139:6833/3174744460] up:boot 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: fsmap 1 up:standby 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:38.565 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:38 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3384324409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6832/3174744460,v1:172.21.15.139:6833/3174744460] up:boot 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: fsmap 1 up:standby 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:38 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3384324409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6832/3174744460,v1:172.21.15.139:6833/3174744460] up:boot 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: fsmap 1 up:standby 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:38.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:38 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:39.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:39 smithi097 ceph-mon[32667]: Deploying daemon mds.k on smithi097 2024-04-25T20:53:39.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:39 smithi139 ceph-mon[33027]: Deploying daemon mds.k on smithi097 2024-04-25T20:53:39.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:39 smithi096 ceph-mon[29288]: Deploying daemon mds.k on smithi097 2024-04-25T20:53:40.285 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 systemd[1]: Starting Ceph mds.k for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:53:40.423 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:53:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:40 smithi097 ceph-mon[32667]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:40.574 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 podman[61956]: 2024-04-25 20:53:40.189366979 +0000 UTC m=+0.017803383 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:53:40.574 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 podman[61956]: 2024-04-25 20:53:40.320695353 +0000 UTC m=+0.149131739 container create d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-25T20:53:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:40 smithi139 ceph-mon[33027]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:40.667 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:40 smithi096 ceph-mon[29288]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:40.863 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 podman[61956]: 2024-04-25 20:53:40.574221703 +0000 UTC m=+0.402658092 container init d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T20:53:40.863 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 podman[61956]: 2024-04-25 20:53:40.576813946 +0000 UTC m=+0.405250343 container start d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:53:40.864 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 ceph-mds[61974]: -- 172.21.15.97:0/246317815 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56267151da40 con 0x5626714f0c00 2024-04-25T20:53:40.864 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k[61970]: starting mds.k at 2024-04-25T20:53:40.864 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 bash[61956]: d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 2024-04-25T20:53:40.864 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 20:53:40 smithi097 systemd[1]: Started Ceph mds.k for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:41.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: Deploying daemon mds.j on smithi096 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6832/2453677033,v1:172.21.15.97:6833/2453677033] up:boot 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: fsmap 2 up:standby 2024-04-25T20:53:42.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:41 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:53:42.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: Deploying daemon mds.j on smithi096 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6832/2453677033,v1:172.21.15.97:6833/2453677033] up:boot 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: fsmap 2 up:standby 2024-04-25T20:53:42.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:41 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: Deploying daemon mds.j on smithi096 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6832/2453677033,v1:172.21.15.97:6833/2453677033] up:boot 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: fsmap 2 up:standby 2024-04-25T20:53:42.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:41 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.429+0000 7f9d5d720700 1 -- 172.21.15.96:0/2840073069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58107990 msgr2=0x7f9d58107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.429+0000 7f9d5d720700 1 --2- 172.21.15.96:0/2840073069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58107990 0x7f9d58107d70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9d48009990 tx=0x7f9d48009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 -- 172.21.15.96:0/2840073069 shutdown_connections 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 --2- 172.21.15.96:0/2840073069 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d58102300 0x7f9d5810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 --2- 172.21.15.96:0/2840073069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58107990 0x7f9d58107d70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 --2- 172.21.15.96:0/2840073069 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 0x7f9d58101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 -- 172.21.15.96:0/2840073069 >> 172.21.15.96:0/2840073069 conn(0x7f9d580788f0 msgr2=0x7f9d58078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:42.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 -- 172.21.15.96:0/2840073069 shutdown_connections 2024-04-25T20:53:42.432 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 -- 172.21.15.96:0/2840073069 wait complete. 2024-04-25T20:53:42.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.430+0000 7f9d5d720700 1 Processor -- start 2024-04-25T20:53:42.433 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 -- start start 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 0x7f9d581980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 0x7f9d58198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d58107990 0x7f9d5819c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58111b60 con 0x7f9d58102300 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d581119e0 con 0x7f9d58101940 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d5d720700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d58111ce0 con 0x7f9d58107990 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 0x7f9d58198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 0x7f9d58198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:48438/0 (socket says 172.21.15.96:48438) 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 -- 172.21.15.96:0/3493148893 learned_addr learned my addr 172.21.15.96:0/3493148893 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d56ffd700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 0x7f9d581980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d577fe700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d58107990 0x7f9d5819c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 -- 172.21.15.96:0/3493148893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d58107990 msgr2=0x7f9d5819c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:42.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d58107990 0x7f9d5819c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 -- 172.21.15.96:0/3493148893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 msgr2=0x7f9d581980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 0x7f9d581980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d567fc700 1 -- 172.21.15.96:0/3493148893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5819d100 con 0x7f9d58102300 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.431+0000 7f9d56ffd700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 0x7f9d581980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.432+0000 7f9d567fc700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 0x7f9d58198630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00d950 tx=0x7f9d4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.432+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c016070 con 0x7f9d58102300 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.432+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5819d3f0 con 0x7f9d58102300 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.432+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5806a720 con 0x7f9d58102300 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.432+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d4c009df0 con 0x7f9d58102300 2024-04-25T20:53:42.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.432+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00f3c0 con 0x7f9d58102300 2024-04-25T20:53:42.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.433+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d58069730 con 0x7f9d58102300 2024-04-25T20:53:42.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.435+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9d4c019070 con 0x7f9d58102300 2024-04-25T20:53:42.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.435+0000 7f9d3ffff700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d400a3c40 0x7f9d400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:42.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.435+0000 7f9d56ffd700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d400a3c40 0x7f9d400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:42.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.435+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f9d4c0cfb50 con 0x7f9d58102300 2024-04-25T20:53:42.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.436+0000 7f9d56ffd700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d400a3c40 0x7f9d400a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800b5c0 tx=0x7f9d48005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:42.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.443+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d4c09c9d0 con 0x7f9d58102300 2024-04-25T20:53:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.694+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d5806b210 con 0x7f9d58102300 2024-04-25T20:53:42.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.694+0000 7f9d3ffff700 1 -- 172.21.15.96:0/3493148893 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f9d4c09c2d0 con 0x7f9d58102300 2024-04-25T20:53:42.695 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:42.696 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T20:53:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d400a3c40 msgr2=0x7f9d400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d400a3c40 0x7f9d400a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800b5c0 tx=0x7f9d48005960 comp rx=0 tx=0).stop 2024-04-25T20:53:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 msgr2=0x7f9d58198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 0x7f9d58198630 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00d950 tx=0x7f9d4c00dc60 comp rx=0 tx=0).stop 2024-04-25T20:53:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 shutdown_connections 2024-04-25T20:53:42.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d400a3c40 0x7f9d400a6100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d58107990 0x7f9d5819c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d58101940 0x7f9d581980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 --2- 172.21.15.96:0/3493148893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d58102300 0x7f9d58198630 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 >> 172.21.15.96:0/3493148893 conn(0x7f9d580788f0 msgr2=0x7f9d581007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 shutdown_connections 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:42.696+0000 7f9d5d720700 1 -- 172.21.15.96:0/3493148893 wait complete. 2024-04-25T20:53:42.698 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 3 2024-04-25T20:53:42.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:42 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3493148893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:43.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:42 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3493148893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:43.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:42 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3493148893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:43.960 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:43 smithi096 ceph-mon[29288]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:44.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:43 smithi097 ceph-mon[32667]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:44.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:43 smithi139 ceph-mon[33027]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:45.499 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 20:53:45 smithi096 systemd[1]: Started Ceph mds.j for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.96:6834/2040913717,v1:172.21.15.96:6835/2040913717] up:boot 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: fsmap 3 up:standby 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: Deploying daemon mds.d on smithi096 2024-04-25T20:53:46.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:46 smithi139 ceph-mon[33027]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.96:6834/2040913717,v1:172.21.15.96:6835/2040913717] up:boot 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: fsmap 3 up:standby 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: Deploying daemon mds.d on smithi096 2024-04-25T20:53:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:46 smithi096 ceph-mon[29288]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:46.593 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:53:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.96:6834/2040913717,v1:172.21.15.96:6835/2040913717] up:boot 2024-04-25T20:53:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: fsmap 3 up:standby 2024-04-25T20:53:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: Deploying daemon mds.d on smithi096 2024-04-25T20:53:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:46 smithi097 ceph-mon[32667]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:46.969 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:48 smithi096 ceph-mon[29288]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:48 smithi097 ceph-mon[32667]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:48 smithi139 ceph-mon[33027]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 -- 172.21.15.96:0/2327384689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030106400 msgr2=0x7fb03010dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 --2- 172.21.15.96:0/2327384689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030106400 0x7fb03010dfd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb02800b0d0 tx=0x7fb02800b3e0 comp rx=0 tx=0).stop 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 -- 172.21.15.96:0/2327384689 shutdown_connections 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 --2- 172.21.15.96:0/2327384689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030106400 0x7fb03010dfd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 --2- 172.21.15.96:0/2327384689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030101320 0x7fb030101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 --2- 172.21.15.96:0/2327384689 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030101cd0 0x7fb030105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 -- 172.21.15.96:0/2327384689 >> 172.21.15.96:0/2327384689 conn(0x7fb030077f40 msgr2=0x7fb0300ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:48.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 -- 172.21.15.96:0/2327384689 shutdown_connections 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.779+0000 7fb034ff9700 1 -- 172.21.15.96:0/2327384689 wait complete. 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 Processor -- start 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 -- start start 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030101320 0x7fb0301980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 0x7fb0301985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030106400 0x7fb03019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb030111b90 con 0x7fb030101cd0 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb030111a10 con 0x7fb030106400 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb034ff9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb030111d10 con 0x7fb030101320 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 0x7fb0301985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 0x7fb0301985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57080/0 (socket says 172.21.15.96:57080) 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 -- 172.21.15.96:0/1008356042 learned_addr learned my addr 172.21.15.96:0/1008356042 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb02e59c700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030101320 0x7fb0301980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:48.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb02ed9d700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030106400 0x7fb03019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 -- 172.21.15.96:0/1008356042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030101320 msgr2=0x7fb0301980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030101320 0x7fb0301980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 -- 172.21.15.96:0/1008356042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030106400 msgr2=0x7fb03019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030106400 0x7fb03019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 -- 172.21.15.96:0/1008356042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb03019d060 con 0x7fb030101cd0 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb02ed9d700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030106400 0x7fb03019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.780+0000 7fb027fff700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 0x7fb0301985e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b500 tx=0x7fb01800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.781+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018015020 con 0x7fb030101cd0 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.781+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb018011ce0 con 0x7fb030101cd0 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.781+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018010d40 con 0x7fb030101cd0 2024-04-25T20:53:48.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.781+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb03019d350 con 0x7fb030101cd0 2024-04-25T20:53:48.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.781+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb03010e8e0 con 0x7fb030101cd0 2024-04-25T20:53:48.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.782+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb030069730 con 0x7fb030101cd0 2024-04-25T20:53:48.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.783+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb018018020 con 0x7fb030101cd0 2024-04-25T20:53:48.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.783+0000 7fb0277fe700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb01c0a3c40 0x7fb01c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:48.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.783+0000 7fb02e59c700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb01c0a3c40 0x7fb01c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:48.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.783+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb0180cfce0 con 0x7fb030101cd0 2024-04-25T20:53:48.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.784+0000 7fb02e59c700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb01c0a3c40 0x7fb01c0a6100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb020005fd0 tx=0x7fb02000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:48.788 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:48.787+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb01801f020 con 0x7fb030101cd0 2024-04-25T20:53:49.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.049+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0301993b0 con 0x7fb030101cd0 2024-04-25T20:53:49.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.049+0000 7fb0277fe700 1 -- 172.21.15.96:0/1008356042 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7fb0301993b0 con 0x7fb030101cd0 2024-04-25T20:53:49.051 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:49.051 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-25T20:53:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb01c0a3c40 msgr2=0x7fb01c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb01c0a3c40 0x7fb01c0a6100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb020005fd0 tx=0x7fb02000a560 comp rx=0 tx=0).stop 2024-04-25T20:53:49.052 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 msgr2=0x7fb0301985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 0x7fb0301985e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb01800b500 tx=0x7fb01800b8c0 comp rx=0 tx=0).stop 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 shutdown_connections 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb01c0a3c40 0x7fb01c0a6100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb030106400 0x7fb03019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb030101320 0x7fb0301980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 --2- 172.21.15.96:0/1008356042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb030101cd0 0x7fb0301985e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 >> 172.21.15.96:0/1008356042 conn(0x7fb030077f40 msgr2=0x7fb0300ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 shutdown_connections 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:49.051+0000 7fb034ff9700 1 -- 172.21.15.96:0/1008356042 wait complete. 2024-04-25T20:53:49.053 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 4 2024-04-25T20:53:49.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:49 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1008356042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:49.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:49 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1008356042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:49.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:49 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1008356042' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:49.750 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 20:53:49 smithi096 systemd[1]: Started Ceph mds.d for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:50.515 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:50 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:50 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:50.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:50 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:51.609 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:51 smithi097 ceph-mon[32667]: Deploying daemon mds.e on smithi097 2024-04-25T20:53:51.609 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:51 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.96:6836/2090633506,v1:172.21.15.96:6837/2090633506] up:boot 2024-04-25T20:53:51.609 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:51 smithi097 ceph-mon[32667]: fsmap 4 up:standby 2024-04-25T20:53:51.609 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:51 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:53:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:51 smithi139 ceph-mon[33027]: Deploying daemon mds.e on smithi097 2024-04-25T20:53:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:51 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.96:6836/2090633506,v1:172.21.15.96:6837/2090633506] up:boot 2024-04-25T20:53:51.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:51 smithi139 ceph-mon[33027]: fsmap 4 up:standby 2024-04-25T20:53:51.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:51 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:53:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:51 smithi096 ceph-mon[29288]: Deploying daemon mds.e on smithi097 2024-04-25T20:53:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:51 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.96:6836/2090633506,v1:172.21.15.96:6837/2090633506] up:boot 2024-04-25T20:53:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:51 smithi096 ceph-mon[29288]: fsmap 4 up:standby 2024-04-25T20:53:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:51 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:53:51.863 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:51 smithi097 systemd[1]: Starting Ceph mds.e for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:53:51.914 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:53:52.166 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:52.283 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:51 smithi097 podman[62267]: 2024-04-25 20:53:51.997150965 +0000 UTC m=+0.105265190 container create 5fd78acc5bdd113464b133d4ba0e00e697fa6e1cea2ab68c0743e63cd23b716b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T20:53:52.283 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 podman[62267]: 2024-04-25 20:53:51.910012868 +0000 UTC m=+0.018127116 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:53:52.283 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 podman[62267]: 2024-04-25 20:53:52.216089375 +0000 UTC m=+0.324203600 container init 5fd78acc5bdd113464b133d4ba0e00e697fa6e1cea2ab68c0743e63cd23b716b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-25T20:53:52.283 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 podman[62267]: 2024-04-25 20:53:52.218827344 +0000 UTC m=+0.326941574 container start 5fd78acc5bdd113464b133d4ba0e00e697fa6e1cea2ab68c0743e63cd23b716b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T20:53:52.284 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 ceph-mds[62287]: -- 172.21.15.97:0/3521256479 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec249b7a40 con 0x55ec2498b000 2024-04-25T20:53:52.284 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e[62282]: starting mds.e at 2024-04-25T20:53:52.284 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 bash[62267]: 5fd78acc5bdd113464b133d4ba0e00e697fa6e1cea2ab68c0743e63cd23b716b 2024-04-25T20:53:52.535 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 20:53:52 smithi097 systemd[1]: Started Ceph mds.e for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:52.535 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:52 smithi097 ceph-mon[32667]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:52.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:52 smithi096 ceph-mon[29288]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:52 smithi139 ceph-mon[33027]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6834/122204709,v1:172.21.15.97:6835/122204709] up:boot 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: fsmap 5 up:standby 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:53.366 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:53 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:53.575 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6834/122204709,v1:172.21.15.97:6835/122204709] up:boot 2024-04-25T20:53:53.575 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: fsmap 5 up:standby 2024-04-25T20:53:53.575 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:53:53.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:53.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:53.576 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:53 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:53.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6834/122204709,v1:172.21.15.97:6835/122204709] up:boot 2024-04-25T20:53:53.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: fsmap 5 up:standby 2024-04-25T20:53:53.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:53:53.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:53.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:53.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:53.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:53 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:53.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 -- 172.21.15.96:0/852671506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec101940 msgr2=0x7f53ec101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:53.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 --2- 172.21.15.96:0/852671506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec101940 0x7f53ec101dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f53e0009990 tx=0x7f53e0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:53.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 -- 172.21.15.96:0/852671506 shutdown_connections 2024-04-25T20:53:53.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 --2- 172.21.15.96:0/852671506 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec102300 0x7f53ec10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 --2- 172.21.15.96:0/852671506 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53ec107990 0x7f53ec107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 --2- 172.21.15.96:0/852671506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec101940 0x7f53ec101dc0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 -- 172.21.15.96:0/852671506 >> 172.21.15.96:0/852671506 conn(0x7f53ec0788f0 msgr2=0x7f53ec078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 -- 172.21.15.96:0/852671506 shutdown_connections 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.772+0000 7f53f170e700 1 -- 172.21.15.96:0/852671506 wait complete. 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 Processor -- start 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 -- start start 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec101940 0x7f53ec199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53ec102300 0x7f53ec19a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 0x7f53ec1a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec111ae0 con 0x7f53ec107990 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec111960 con 0x7f53ec102300 2024-04-25T20:53:53.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53f170e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f53ec111c60 con 0x7f53ec101940 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 0x7f53ec1a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53eb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 0x7f53ec1a4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57122/0 (socket says 172.21.15.96:57122) 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53eb7fe700 1 -- 172.21.15.96:0/1087579276 learned_addr learned my addr 172.21.15.96:0/1087579276 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53ea7fc700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53ec102300 0x7f53ec19a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.773+0000 7f53eaffd700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec101940 0x7f53ec199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53eb7fe700 1 -- 172.21.15.96:0/1087579276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec101940 msgr2=0x7f53ec199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53eb7fe700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec101940 0x7f53ec199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53eb7fe700 1 -- 172.21.15.96:0/1087579276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53ec102300 msgr2=0x7f53ec19a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53eb7fe700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53ec102300 0x7f53ec19a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53eb7fe700 1 -- 172.21.15.96:0/1087579276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53ec1a5420 con 0x7f53ec107990 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53eb7fe700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 0x7f53ec1a4ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c900 tx=0x7f53dc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc007b00 con 0x7f53ec107990 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53dc00ce50 con 0x7f53ec107990 2024-04-25T20:53:53.783 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53ec1a56b0 con 0x7f53ec107990 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53dc018660 con 0x7f53ec107990 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.774+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53ec1a5bc0 con 0x7f53ec107990 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.775+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53ec103060 con 0x7f53ec107990 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.776+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f53dc00f450 con 0x7f53ec107990 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.777+0000 7f53d3fff700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53d40a3c40 0x7f53d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.777+0000 7f53eaffd700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53d40a3c40 0x7f53d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.777+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f53dc0cfff0 con 0x7f53ec107990 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.777+0000 7f53eaffd700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53d40a3c40 0x7f53d40a6100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009c90 tx=0x7f53e4009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:53.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:53.781+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f53dc014030 con 0x7f53ec107990 2024-04-25T20:53:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.025+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53ec02df80 con 0x7f53ec107990 2024-04-25T20:53:54.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.025+0000 7f53d3fff700 1 -- 172.21.15.96:0/1087579276 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7f53dc09ce70 con 0x7f53ec107990 2024-04-25T20:53:54.027 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:54.027 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.027+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53d40a3c40 msgr2=0x7f53d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.027+0000 7f53f170e700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53d40a3c40 0x7f53d40a6100 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f53e4009c90 tx=0x7f53e4009040 comp rx=0 tx=0).stop 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.027+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 msgr2=0x7f53ec1a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.027+0000 7f53f170e700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 0x7f53ec1a4ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f53dc00c900 tx=0x7f53dc00cc10 comp rx=0 tx=0).stop 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 shutdown_connections 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53d40a3c40 0x7f53d40a6100 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53ec107990 0x7f53ec1a4ee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f53ec101940 0x7f53ec199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 --2- 172.21.15.96:0/1087579276 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f53ec102300 0x7f53ec19a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 >> 172.21.15.96:0/1087579276 conn(0x7f53ec0788f0 msgr2=0x7f53ec0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 shutdown_connections 2024-04-25T20:53:54.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:54.028+0000 7f53f170e700 1 -- 172.21.15.96:0/1087579276 wait complete. 2024-04-25T20:53:54.030 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 6 2024-04-25T20:53:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:54 smithi139 ceph-mon[33027]: Deploying daemon mds.f on smithi139 2024-04-25T20:53:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:54 smithi139 ceph-mon[33027]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:54.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:54 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1087579276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:54 smithi096 ceph-mon[29288]: Deploying daemon mds.f on smithi139 2024-04-25T20:53:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:54 smithi096 ceph-mon[29288]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:54.686 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:54 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1087579276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:54 smithi097 ceph-mon[32667]: Deploying daemon mds.f on smithi139 2024-04-25T20:53:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:54 smithi097 ceph-mon[32667]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:54.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:54 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1087579276' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:55.372 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 20:53:55 smithi139 systemd[1]: Started Ceph mds.f for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:56.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:56.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:56.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: Deploying daemon mds.c on smithi139 2024-04-25T20:53:56.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6834/1172572610,v1:172.21.15.139:6835/1172572610] up:boot 2024-04-25T20:53:56.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: fsmap 6 up:standby 2024-04-25T20:53:56.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:56 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:53:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: Deploying daemon mds.c on smithi139 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6834/1172572610,v1:172.21.15.139:6835/1172572610] up:boot 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: fsmap 6 up:standby 2024-04-25T20:53:56.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:56 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: Deploying daemon mds.c on smithi139 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6834/1172572610,v1:172.21.15.139:6835/1172572610] up:boot 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: fsmap 6 up:standby 2024-04-25T20:53:56.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:56 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:53:56.714 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:53:56.968 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:53:57.616 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 systemd[1]: Starting Ceph mds.c for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:53:57.868 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 podman[63278]: 2024-04-25 20:53:57.518718203 +0000 UTC m=+0.017962491 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:53:57.869 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 podman[63278]: 2024-04-25 20:53:57.632209748 +0000 UTC m=+0.131454068 container create c1a2b34257bdca2dc2927c0ff96279b536b337b26f23eb15b013f38f946d953e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T20:53:57.869 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 podman[63278]: 2024-04-25 20:53:57.842713584 +0000 UTC m=+0.341957859 container init c1a2b34257bdca2dc2927c0ff96279b536b337b26f23eb15b013f38f946d953e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:53:57.869 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 podman[63278]: 2024-04-25 20:53:57.845363528 +0000 UTC m=+0.344607811 container start c1a2b34257bdca2dc2927c0ff96279b536b337b26f23eb15b013f38f946d953e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True) 2024-04-25T20:53:58.122 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 ceph-mds[63297]: -- 172.21.15.139:0/2257712432 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f202ed3a40 con 0x55f202ea6c00 2024-04-25T20:53:58.122 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: starting mds.c at 2024-04-25T20:53:58.123 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 bash[63278]: c1a2b34257bdca2dc2927c0ff96279b536b337b26f23eb15b013f38f946d953e 2024-04-25T20:53:58.123 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 20:53:57 smithi139 systemd[1]: Started Ceph mds.c for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:53:58.400 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:58.400 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.400 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.401 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.401 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:58.401 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:58.401 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:58 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:58.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:58 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:53:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:53:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:53:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:53:58.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:58 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:53:58.637 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.636+0000 7fb415a48700 1 -- 172.21.15.96:0/970679157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101013e0 msgr2=0x7fb410101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.636+0000 7fb415a48700 1 --2- 172.21.15.96:0/970679157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101013e0 0x7fb410101860 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb404009990 tx=0x7fb404009ca0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 -- 172.21.15.96:0/970679157 shutdown_connections 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 --2- 172.21.15.96:0/970679157 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb410101da0 0x7fb41010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 --2- 172.21.15.96:0/970679157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb410107b00 0x7fb410107ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 --2- 172.21.15.96:0/970679157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4101013e0 0x7fb410101860 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 -- 172.21.15.96:0/970679157 >> 172.21.15.96:0/970679157 conn(0x7fb410076c60 msgr2=0x7fb410077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 -- 172.21.15.96:0/970679157 shutdown_connections 2024-04-25T20:53:58.638 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 -- 172.21.15.96:0/970679157 wait complete. 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 Processor -- start 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.637+0000 7fb415a48700 1 -- start start 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 0x7fb4101990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb410101da0 0x7fb410199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb410107b00 0x7fb41019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112c50 con 0x7fb4101013e0 2024-04-25T20:53:58.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112ad0 con 0x7fb410107b00 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb410112dd0 con 0x7fb410101da0 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 0x7fb4101990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 0x7fb4101990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37596/0 (socket says 172.21.15.96:37596) 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 -- 172.21.15.96:0/623117232 learned_addr learned my addr 172.21.15.96:0/623117232 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40e7fc700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb410101da0 0x7fb410199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40f7fe700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb410107b00 0x7fb41019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 -- 172.21.15.96:0/623117232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb410101da0 msgr2=0x7fb410199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb410101da0 0x7fb410199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 -- 172.21.15.96:0/623117232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb410107b00 msgr2=0x7fb41019d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb410107b00 0x7fb41019d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 -- 172.21.15.96:0/623117232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb41019e0a0 con 0x7fb4101013e0 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40f7fe700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb410107b00 0x7fb41019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb40effd700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 0x7fb4101990e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb40000ea50 tx=0x7fb40000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:58.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb40000cbd0 con 0x7fb4101013e0 2024-04-25T20:53:58.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb41019e390 con 0x7fb4101013e0 2024-04-25T20:53:58.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb40000cd70 con 0x7fb4101013e0 2024-04-25T20:53:58.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.638+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb400018750 con 0x7fb4101013e0 2024-04-25T20:53:58.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.639+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4101a5c10 con 0x7fb4101013e0 2024-04-25T20:53:58.642 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.639+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb410069730 con 0x7fb4101013e0 2024-04-25T20:53:58.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.640+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb40001f020 con 0x7fb4101013e0 2024-04-25T20:53:58.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.641+0000 7fb414a46700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb3f80a3c40 0x7fb3f80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:53:58.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.641+0000 7fb40e7fc700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb3f80a3c40 0x7fb3f80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:53:58.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.641+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fb4000d0190 con 0x7fb4101013e0 2024-04-25T20:53:58.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.642+0000 7fb40e7fc700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb3f80a3c40 0x7fb3f80a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb40400b2e0 tx=0x7fb404005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:53:58.647 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.646+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb400053ff0 con 0x7fb4101013e0 2024-04-25T20:53:58.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.889+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb41002df80 con 0x7fb4101013e0 2024-04-25T20:53:58.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.889+0000 7fb414a46700 1 -- 172.21.15.96:0/623117232 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6343 (secure 0 0 0) 0x7fb40009c9c0 con 0x7fb4101013e0 2024-04-25T20:53:58.891 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:53:58.891 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb3f80a3c40 msgr2=0x7fb3f80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb3f80a3c40 0x7fb3f80a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fb40400b2e0 tx=0x7fb404005960 comp rx=0 tx=0).stop 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 msgr2=0x7fb4101990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 0x7fb4101990e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb40000ea50 tx=0x7fb40000ed60 comp rx=0 tx=0).stop 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 shutdown_connections 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb3f80a3c40 0x7fb3f80a6100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb410107b00 0x7fb41019d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4101013e0 0x7fb4101990e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 --2- 172.21.15.96:0/623117232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb410101da0 0x7fb410199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 >> 172.21.15.96:0/623117232 conn(0x7fb410076c60 msgr2=0x7fb410100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 shutdown_connections 2024-04-25T20:53:58.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:53:58.892+0000 7fb415a48700 1 -- 172.21.15.96:0/623117232 wait complete. 2024-04-25T20:53:58.894 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 8 2024-04-25T20:53:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:59 smithi096 ceph-mon[29288]: Deploying daemon mds.h on smithi097 2024-04-25T20:53:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:59 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6836/57160731,v1:172.21.15.139:6837/57160731] up:boot 2024-04-25T20:53:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:59 smithi096 ceph-mon[29288]: fsmap 7 up:standby 2024-04-25T20:53:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:59 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:53:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:53:59 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/623117232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:59.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:59 smithi097 ceph-mon[32667]: Deploying daemon mds.h on smithi097 2024-04-25T20:53:59.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:59 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6836/57160731,v1:172.21.15.139:6837/57160731] up:boot 2024-04-25T20:53:59.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:59 smithi097 ceph-mon[32667]: fsmap 7 up:standby 2024-04-25T20:53:59.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:59 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:53:59.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:53:59 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/623117232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:53:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:59 smithi139 ceph-mon[33027]: Deploying daemon mds.h on smithi097 2024-04-25T20:53:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:59 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6836/57160731,v1:172.21.15.139:6837/57160731] up:boot 2024-04-25T20:53:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:59 smithi139 ceph-mon[33027]: fsmap 7 up:standby 2024-04-25T20:53:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:59 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:53:59.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:53:59 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/623117232' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:00.245 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 systemd[1]: Starting Ceph mds.h for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:54:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:00 smithi096 ceph-mon[29288]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:00.530 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:00 smithi097 ceph-mon[32667]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:00.530 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 podman[62585]: 2024-04-25 20:54:00.289991048 +0000 UTC m=+0.018762800 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:54:00.530 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 podman[62585]: 2024-04-25 20:54:00.408207114 +0000 UTC m=+0.136978866 container create f21fa8adbfc8ced7e7c593026aceedb239b96a91cd9cf45e89c5b33703a49d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T20:54:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:00 smithi139 ceph-mon[33027]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:01.113 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 podman[62585]: 2024-04-25 20:54:00.635605928 +0000 UTC m=+0.364377684 container init f21fa8adbfc8ced7e7c593026aceedb239b96a91cd9cf45e89c5b33703a49d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-25T20:54:01.114 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 podman[62585]: 2024-04-25 20:54:00.638466671 +0000 UTC m=+0.367238431 container start f21fa8adbfc8ced7e7c593026aceedb239b96a91cd9cf45e89c5b33703a49d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-25T20:54:01.114 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 ceph-mds[62604]: -- 172.21.15.97:0/2326320823 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d1a24a1a40 con 0x55d1a2474800 2024-04-25T20:54:01.114 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h[62600]: starting mds.h at 2024-04-25T20:54:01.114 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 bash[62585]: f21fa8adbfc8ced7e7c593026aceedb239b96a91cd9cf45e89c5b33703a49d75 2024-04-25T20:54:01.114 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 20:54:00 smithi097 systemd[1]: Started Ceph mds.h for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:54:01.546 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:54:01.797 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:01.919 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: Deploying daemon mds.i on smithi139 2024-04-25T20:54:01.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:01.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] up:boot 2024-04-25T20:54:01.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: fsmap 8 up:standby 2024-04-25T20:54:01.920 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:01 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:54:02.107 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:02.107 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:02.107 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:02.107 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: Deploying daemon mds.i on smithi139 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] up:boot 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: fsmap 8 up:standby 2024-04-25T20:54:02.108 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:01 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:54:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:02.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:02.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: Deploying daemon mds.i on smithi139 2024-04-25T20:54:02.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:02.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] up:boot 2024-04-25T20:54:02.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: fsmap 8 up:standby 2024-04-25T20:54:02.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:01 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:54:03.123 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:02 smithi139 systemd[1]: Starting Ceph mds.i for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:54:03.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 -- 172.21.15.96:0/1039099599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f68108a20 msgr2=0x7f6f68108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:03.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/1039099599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f68108a20 0x7f6f68108e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6f580099c0 tx=0x7f6f58009cd0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 -- 172.21.15.96:0/1039099599 shutdown_connections 2024-04-25T20:54:03.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/1039099599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f6806bba0 0x7f6f6810f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.374 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/1039099599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f68108a20 0x7f6f68108e00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/1039099599 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 0x7f6f6806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 -- 172.21.15.96:0/1039099599 >> 172.21.15.96:0/1039099599 conn(0x7f6f680779b0 msgr2=0x7f6f68077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 -- 172.21.15.96:0/1039099599 shutdown_connections 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 -- 172.21.15.96:0/1039099599 wait complete. 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.371+0000 7f6f6d62b700 1 Processor -- start 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 -- start start 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 0x7f6f68101a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 0x7f6f68101f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f68108a20 0x7f6f681024c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:03.377 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68112bf0 con 0x7f6f6806bba0 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68112a70 con 0x7f6f68108a20 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f6d62b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f68112d70 con 0x7f6f6806b1e0 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f5e7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 0x7f6f68101f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 0x7f6f68101a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f5e7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 0x7f6f68101f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37608/0 (socket says 172.21.15.96:37608) 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f5e7fc700 1 -- 172.21.15.96:0/2063828417 learned_addr learned my addr 172.21.15.96:0/2063828417 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f677fe700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f68108a20 0x7f6f681024c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 -- 172.21.15.96:0/2063828417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f68108a20 msgr2=0x7f6f681024c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f68108a20 0x7f6f681024c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 -- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 msgr2=0x7f6f68101f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 0x7f6f68101f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 -- 172.21.15.96:0/2063828417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f681a7610 con 0x7f6f6806b1e0 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f5e7fc700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 0x7f6f68101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:54:03.378 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.372+0000 7f6f66ffd700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 0x7f6f68101a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009f50 tx=0x7f6f580047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.373+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58027070 con 0x7f6f6806b1e0 2024-04-25T20:54:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.373+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f681a78a0 con 0x7f6f6806b1e0 2024-04-25T20:54:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.373+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f58003f20 con 0x7f6f6806b1e0 2024-04-25T20:54:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.373+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58022bd0 con 0x7f6f6806b1e0 2024-04-25T20:54:03.379 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.373+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f681a7d50 con 0x7f6f6806b1e0 2024-04-25T20:54:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.374+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f681041a0 con 0x7f6f6806b1e0 2024-04-25T20:54:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.375+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6f5802a070 con 0x7f6f6806b1e0 2024-04-25T20:54:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.375+0000 7f6f64ff9700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f540a3c40 0x7f6f540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.375+0000 7f6f5e7fc700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f540a3c40 0x7f6f540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.375+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f6f580d0de0 con 0x7f6f6806b1e0 2024-04-25T20:54:03.380 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.375+0000 7f6f5e7fc700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f540a3c40 0x7f6f540a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f68078020 tx=0x7f6f50007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:03.381 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.380+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f5805eff0 con 0x7f6f6806b1e0 2024-04-25T20:54:03.396 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 podman[63585]: 2024-04-25 20:54:03.028683047 +0000 UTC m=+0.018775606 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:54:03.396 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 podman[63585]: 2024-04-25 20:54:03.126311603 +0000 UTC m=+0.116404150 container create d14a0dc08e86ede370eb50aaf72e15160c01bc938d2fb656cdc52bab94e8851b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T20:54:03.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.629+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f6802df80 con 0x7f6f6806b1e0 2024-04-25T20:54:03.630 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.629+0000 7f6f64ff9700 1 -- 172.21.15.96:0/2063828417 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7162 (secure 0 0 0) 0x7f6f58013070 con 0x7f6f6806b1e0 2024-04-25T20:54:03.630 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:03.631 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-25T20:54:03.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f540a3c40 msgr2=0x7f6f540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:03.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f540a3c40 0x7f6f540a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6f68078020 tx=0x7f6f50007480 comp rx=0 tx=0).stop 2024-04-25T20:54:03.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 msgr2=0x7f6f68101a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:03.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 0x7f6f68101a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6f58009f50 tx=0x7f6f580047b0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.633 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 shutdown_connections 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f540a3c40 0x7f6f540a6100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f68108a20 0x7f6f681024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f6806b1e0 0x7f6f68101a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 --2- 172.21.15.96:0/2063828417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f6806bba0 0x7f6f68101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 >> 172.21.15.96:0/2063828417 conn(0x7f6f680779b0 msgr2=0x7f6f681006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 shutdown_connections 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:03.632+0000 7f6f6d62b700 1 -- 172.21.15.96:0/2063828417 wait complete. 2024-04-25T20:54:03.634 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 9 2024-04-25T20:54:03.873 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 podman[63585]: 2024-04-25 20:54:03.3957303 +0000 UTC m=+0.385822846 container init d14a0dc08e86ede370eb50aaf72e15160c01bc938d2fb656cdc52bab94e8851b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T20:54:03.873 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 podman[63585]: 2024-04-25 20:54:03.398872878 +0000 UTC m=+0.388965427 container start d14a0dc08e86ede370eb50aaf72e15160c01bc938d2fb656cdc52bab94e8851b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T20:54:03.873 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 ceph-mds[63604]: -- 172.21.15.139:0/1697632526 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600c4aada40 con 0x5600c4a80c00 2024-04-25T20:54:03.873 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i[63600]: starting mds.i at 2024-04-25T20:54:03.873 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 bash[63585]: d14a0dc08e86ede370eb50aaf72e15160c01bc938d2fb656cdc52bab94e8851b 2024-04-25T20:54:03.873 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 20:54:03 smithi139 systemd[1]: Started Ceph mds.i for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:04.541 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:04 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2063828417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:04.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:04.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:04.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:04 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2063828417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:04.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:04 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2063828417' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:05.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:05 smithi096 ceph-mon[29288]: Deploying daemon mds.g on smithi096 2024-04-25T20:54:05.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:05 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] up:boot 2024-04-25T20:54:05.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:05 smithi096 ceph-mon[29288]: fsmap 9 up:standby 2024-04-25T20:54:05.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:05 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:54:05.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:05 smithi097 ceph-mon[32667]: Deploying daemon mds.g on smithi096 2024-04-25T20:54:05.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:05 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] up:boot 2024-04-25T20:54:05.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:05 smithi097 ceph-mon[32667]: fsmap 9 up:standby 2024-04-25T20:54:05.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:05 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:54:05.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:05 smithi139 ceph-mon[33027]: Deploying daemon mds.g on smithi096 2024-04-25T20:54:05.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:05 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] up:boot 2024-04-25T20:54:05.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:05 smithi139 ceph-mon[33027]: fsmap 9 up:standby 2024-04-25T20:54:05.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:05 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:54:06.250 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 systemd[1]: Starting Ceph mds.g for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:54:06.282 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:54:06.535 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:06 smithi096 ceph-mon[29288]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:06.536 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 podman[70595]: 2024-04-25 20:54:06.184907612 +0000 UTC m=+0.018299725 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:54:06.536 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 podman[70595]: 2024-04-25 20:54:06.307744899 +0000 UTC m=+0.141137058 container create 2984ac24830070b439e744497e79d18986af120acb5484207ec78a050f76cdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-25T20:54:06.536 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 podman[70595]: 2024-04-25 20:54:06.535406411 +0000 UTC m=+0.368798505 container init 2984ac24830070b439e744497e79d18986af120acb5484207ec78a050f76cdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T20:54:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:06 smithi097 ceph-mon[32667]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:06 smithi139 ceph-mon[33027]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:06.689 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:06.812 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 podman[70595]: 2024-04-25 20:54:06.539234124 +0000 UTC m=+0.372626223 container start 2984ac24830070b439e744497e79d18986af120acb5484207ec78a050f76cdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-25T20:54:06.812 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 ceph-mds[70642]: -- 172.21.15.96:0/1357779529 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d56843fa40 con 0x55d568413000 2024-04-25T20:54:06.812 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g[70627]: starting mds.g at 2024-04-25T20:54:06.813 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 bash[70595]: 2984ac24830070b439e744497e79d18986af120acb5484207ec78a050f76cdb8 2024-04-25T20:54:06.813 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 20:54:06 smithi096 systemd[1]: Started Ceph mds.g for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: Deploying daemon mds.a on smithi096 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:07.785 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.96:6838/2508705262,v1:172.21.15.96:6839/2508705262] up:boot 2024-04-25T20:54:07.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: fsmap 10 up:standby 2024-04-25T20:54:07.786 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:07 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:54:08.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: Deploying daemon mds.a on smithi096 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.96:6838/2508705262,v1:172.21.15.96:6839/2508705262] up:boot 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: fsmap 10 up:standby 2024-04-25T20:54:08.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:07 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:54:08.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: Deploying daemon mds.a on smithi096 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.96:6838/2508705262,v1:172.21.15.96:6839/2508705262] up:boot 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: fsmap 10 up:standby 2024-04-25T20:54:08.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:07 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 -- 172.21.15.96:0/382030171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 msgr2=0x7f799410e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 --2- 172.21.15.96:0/382030171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f799410e7f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f798c009990 tx=0x7f798c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 -- 172.21.15.96:0/382030171 shutdown_connections 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 --2- 172.21.15.96:0/382030171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f799410e7f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 --2- 172.21.15.96:0/382030171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994107920 0x7f7994107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 --2- 172.21.15.96:0/382030171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994101970 0x7f7994101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 -- 172.21.15.96:0/382030171 >> 172.21.15.96:0/382030171 conn(0x7f79940788f0 msgr2=0x7f7994078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:08.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 -- 172.21.15.96:0/382030171 shutdown_connections 2024-04-25T20:54:08.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.381+0000 7f799c469700 1 -- 172.21.15.96:0/382030171 wait complete. 2024-04-25T20:54:08.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 Processor -- start 2024-04-25T20:54:08.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 -- start start 2024-04-25T20:54:08.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994101970 0x7f7994199380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:08.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f79941998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994107920 0x7f799419dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111bc0 con 0x7f7994102330 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111a40 con 0x7f7994107920 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f799c469700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7994111d40 con 0x7f7994101970 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.382+0000 7f7999a04700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f79941998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f79941998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35964/0 (socket says 172.21.15.96:35964) 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 -- 172.21.15.96:0/1864454179 learned_addr learned my addr 172.21.15.96:0/1864454179 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f799a205700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994101970 0x7f7994199380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f799aa06700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994107920 0x7f799419dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 -- 172.21.15.96:0/1864454179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994101970 msgr2=0x7f7994199380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994101970 0x7f7994199380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 -- 172.21.15.96:0/1864454179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994107920 msgr2=0x7f799419dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:08.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994107920 0x7f799419dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 -- 172.21.15.96:0/1864454179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799419e340 con 0x7f7994102330 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f799a205700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994101970 0x7f7994199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f799aa06700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994107920 0x7f799419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f7999a04700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f79941998c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f798800ba50 tx=0x7f798800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7988019070 con 0x7f7994102330 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7988014440 con 0x7f7994102330 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79880075d0 con 0x7f7994102330 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799419e630 con 0x7f7994102330 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.383+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79941a5eb0 con 0x7f7994102330 2024-04-25T20:54:08.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.384+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7994069730 con 0x7f7994102330 2024-04-25T20:54:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.385+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f798801c070 con 0x7f7994102330 2024-04-25T20:54:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.386+0000 7f79937fe700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.386+0000 7f799a205700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.386+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f79880cf880 con 0x7f7994102330 2024-04-25T20:54:08.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.386+0000 7f799a205700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79840099d0 tx=0x7f7984008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:08.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.390+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f798809c700 con 0x7f7994102330 2024-04-25T20:54:08.640 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.639+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799419a690 con 0x7f7994102330 2024-04-25T20:54:08.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.640+0000 7f79937fe700 1 -- 172.21.15.96:0/1864454179 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8818 (secure 0 0 0) 0x7f798809c000 con 0x7f7994102330 2024-04-25T20:54:08.641 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:08.641 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 msgr2=0x7f797c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f79840099d0 tx=0x7f7984008040 comp rx=0 tx=0).stop 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 msgr2=0x7f79941998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f79941998c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f798800ba50 tx=0x7f798800be10 comp rx=0 tx=0).stop 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 shutdown_connections 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7994107920 0x7f799419dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.643 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7994101970 0x7f7994199380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 --2- 172.21.15.96:0/1864454179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7994102330 0x7f79941998c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 >> 172.21.15.96:0/1864454179 conn(0x7f79940788f0 msgr2=0x7f799406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 shutdown_connections 2024-04-25T20:54:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:08.642+0000 7f799c469700 1 -- 172.21.15.96:0/1864454179 wait complete. 2024-04-25T20:54:08.644 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 11 2024-04-25T20:54:08.984 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:08 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1864454179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:09.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:08 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1864454179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:09.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:08 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1864454179' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:09.886 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:09 smithi096 ceph-mon[29288]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:10.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:09 smithi097 ceph-mon[32667]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:09 smithi139 ceph-mon[33027]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:10.999 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 20:54:10 smithi096 systemd[1]: Started Ceph mds.a for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:54:11.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.96:6840/3884645906,v1:172.21.15.96:6841/3884645906] up:boot 2024-04-25T20:54:11.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: fsmap 11 up:standby 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: Deploying daemon mds.b on smithi097 2024-04-25T20:54:12.000 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:11 smithi096 ceph-mon[29288]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:12.037 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:54:12.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.96:6840/3884645906,v1:172.21.15.96:6841/3884645906] up:boot 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: fsmap 11 up:standby 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: Deploying daemon mds.b on smithi097 2024-04-25T20:54:12.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:11 smithi097 ceph-mon[32667]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:12.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.96:6840/3884645906,v1:172.21.15.96:6841/3884645906] up:boot 2024-04-25T20:54:12.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: fsmap 11 up:standby 2024-04-25T20:54:12.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: Deploying daemon mds.b on smithi097 2024-04-25T20:54:12.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:11 smithi139 ceph-mon[33027]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:12.291 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:13.113 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:12 smithi097 systemd[1]: Starting Ceph mds.b for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T20:54:13.472 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 podman[62894]: 2024-04-25 20:54:13.110356969 +0000 UTC m=+0.017751813 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T20:54:13.473 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 podman[62894]: 2024-04-25 20:54:13.219579484 +0000 UTC m=+0.126974302 container create 37504cce1767de3997a9885a26682920bf853c47876783fa1d6e1d83d9b223c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T20:54:13.473 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 podman[62894]: 2024-04-25 20:54:13.44687981 +0000 UTC m=+0.354274627 container init 37504cce1767de3997a9885a26682920bf853c47876783fa1d6e1d83d9b223c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-25T20:54:13.473 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 podman[62894]: 2024-04-25 20:54:13.44943232 +0000 UTC m=+0.356827138 container start 37504cce1767de3997a9885a26682920bf853c47876783fa1d6e1d83d9b223c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T20:54:13.863 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 ceph-mds[62912]: -- 172.21.15.97:0/2782179032 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ceaededa40 con 0x55ceaedc0800 2024-04-25T20:54:13.863 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: starting mds.b at 2024-04-25T20:54:13.863 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 bash[62894]: 37504cce1767de3997a9885a26682920bf853c47876783fa1d6e1d83d9b223c1 2024-04-25T20:54:13.863 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:13 smithi097 systemd[1]: Started Ceph mds.b for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T20:54:13.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 -- 172.21.15.96:0/212975608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec06b980 msgr2=0x7f27ec10f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:13.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 --2- 172.21.15.96:0/212975608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec06b980 0x7f27ec10f8f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f27e400b0a0 tx=0x7f27e400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 -- 172.21.15.96:0/212975608 shutdown_connections 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 --2- 172.21.15.96:0/212975608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec06b980 0x7f27ec10f8f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 --2- 172.21.15.96:0/212975608 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec108a20 0x7f27ec108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 --2- 172.21.15.96:0/212975608 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 0x7f27ec06b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 -- 172.21.15.96:0/212975608 >> 172.21.15.96:0/212975608 conn(0x7f27ec0788b0 msgr2=0x7f27ec078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 -- 172.21.15.96:0/212975608 shutdown_connections 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.978+0000 7f27f20d7700 1 -- 172.21.15.96:0/212975608 wait complete. 2024-04-25T20:54:13.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 Processor -- start 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 -- start start 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 0x7f27ec19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec06b980 0x7f27ec19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 0x7f27ec19fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec112be0 con 0x7f27ec108a20 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec112a60 con 0x7f27ec06b980 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27f20d7700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec112d60 con 0x7f27ec06afe0 2024-04-25T20:54:13.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27ebfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 0x7f27ec19fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 0x7f27ec19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27eaffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec06b980 0x7f27ec19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27ebfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 0x7f27ec19fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35994/0 (socket says 172.21.15.96:35994) 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27eb7fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 0x7f27ec19b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:49658/0 (socket says 172.21.15.96:49658) 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.979+0000 7f27ebfff700 1 -- 172.21.15.96:0/2843372696 learned_addr learned my addr 172.21.15.96:0/2843372696 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27ebfff700 1 -- 172.21.15.96:0/2843372696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 msgr2=0x7f27ec19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27ebfff700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 0x7f27ec19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27ebfff700 1 -- 172.21.15.96:0/2843372696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec06b980 msgr2=0x7f27ec19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27ebfff700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec06b980 0x7f27ec19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27ebfff700 1 -- 172.21.15.96:0/2843372696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec1a0270 con 0x7f27ec108a20 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27eaffd700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec06b980 0x7f27ec19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27ebfff700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 0x7f27ec19fb30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27e400b780 tx=0x7f27e4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4021070 con 0x7f27ec108a20 2024-04-25T20:54:13.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e4007e30 con 0x7f27ec108a20 2024-04-25T20:54:13.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec1a0500 con 0x7f27ec108a20 2024-04-25T20:54:13.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e4004700 con 0x7f27ec108a20 2024-04-25T20:54:13.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.980+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec1a7de0 con 0x7f27ec108a20 2024-04-25T20:54:13.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.981+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27ec069730 con 0x7f27ec108a20 2024-04-25T20:54:13.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.982+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f27e4007980 con 0x7f27ec108a20 2024-04-25T20:54:13.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.983+0000 7f27e8ff9700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27d40a3c40 0x7f27d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:13.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.983+0000 7f27eb7fe700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27d40a3c40 0x7f27d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:13.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.983+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f27e400f020 con 0x7f27ec108a20 2024-04-25T20:54:13.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.983+0000 7f27eb7fe700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27d40a3c40 0x7f27d40a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009be0 tx=0x7f27e0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:13.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:13.987+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f27e402a020 con 0x7f27ec108a20 2024-04-25T20:54:14.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.244+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f27ec02df80 con 0x7f27ec108a20 2024-04-25T20:54:14.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.245+0000 7f27e8ff9700 1 -- 172.21.15.96:0/2843372696 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9644 (secure 0 0 0) 0x7f27e409e5e0 con 0x7f27ec108a20 2024-04-25T20:54:14.245 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:14.245 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-25T20:54:14.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.247+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27d40a3c40 msgr2=0x7f27d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.247+0000 7f27f20d7700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27d40a3c40 0x7f27d40a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009be0 tx=0x7f27e0009260 comp rx=0 tx=0).stop 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.247+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 msgr2=0x7f27ec19fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.247+0000 7f27f20d7700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 0x7f27ec19fb30 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f27e400b780 tx=0x7f27e4004d00 comp rx=0 tx=0).stop 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 shutdown_connections 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f27d40a3c40 0x7f27d40a6100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec108a20 0x7f27ec19fb30 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec06afe0 0x7f27ec19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 --2- 172.21.15.96:0/2843372696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec06b980 0x7f27ec19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 >> 172.21.15.96:0/2843372696 conn(0x7f27ec0788b0 msgr2=0x7f27ec10de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 shutdown_connections 2024-04-25T20:54:14.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:14.248+0000 7f27f20d7700 1 -- 172.21.15.96:0/2843372696 wait complete. 2024-04-25T20:54:14.249 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 12 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:14 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2843372696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:14.557 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:14.557 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.557 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.558 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.558 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.558 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.558 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:14.558 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.558 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:14 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2843372696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:14.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:14 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2843372696' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:15.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:15 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] up:boot 2024-04-25T20:54:15.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:15 smithi096 ceph-mon[29288]: fsmap 12 up:standby 2024-04-25T20:54:15.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:15 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:54:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:15 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] up:boot 2024-04-25T20:54:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:15 smithi097 ceph-mon[32667]: fsmap 12 up:standby 2024-04-25T20:54:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:15 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:54:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:15 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] up:boot 2024-04-25T20:54:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:15 smithi139 ceph-mon[33027]: fsmap 12 up:standby 2024-04-25T20:54:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:15 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:54:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:16 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:16 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:16.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:16 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:54:16.881 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:54:17.620 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:18.416 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:18 smithi096 ceph-mon[29288]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:18.416 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.416 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.416 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.416 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:18 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:18 smithi097 ceph-mon[32667]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:18 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:18 smithi139 ceph-mon[33027]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:18.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:18 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:19.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 -- 172.21.15.96:0/2847632439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69006a750 msgr2=0x7fe69006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 --2- 172.21.15.96:0/2847632439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69006a750 0x7fe69006abb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe680009990 tx=0x7fe680009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 -- 172.21.15.96:0/2847632439 shutdown_connections 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 --2- 172.21.15.96:0/2847632439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe69006b0f0 0x7fe69010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 --2- 172.21.15.96:0/2847632439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe690108a30 0x7fe690108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 --2- 172.21.15.96:0/2847632439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69006a750 0x7fe69006abb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 -- 172.21.15.96:0/2847632439 >> 172.21.15.96:0/2847632439 conn(0x7fe6900788f0 msgr2=0x7fe690078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:19.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.202+0000 7fe6987ed700 1 -- 172.21.15.96:0/2847632439 shutdown_connections 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.203+0000 7fe6987ed700 1 -- 172.21.15.96:0/2847632439 wait complete. 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.203+0000 7fe6987ed700 1 Processor -- start 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.203+0000 7fe6987ed700 1 -- start start 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.203+0000 7fe6987ed700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe69006b0f0 0x7fe6901993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.203+0000 7fe6987ed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690108a30 0x7fe690199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.203+0000 7fe6987ed700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 0x7fe69019e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6987ed700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe690112c20 con 0x7fe69019dcc0 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6987ed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe690112aa0 con 0x7fe690108a30 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6987ed700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe690112da0 con 0x7fe69006b0f0 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 0x7fe69019e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 0x7fe69019e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53914/0 (socket says 172.21.15.96:53914) 2024-04-25T20:54:19.207 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 -- 172.21.15.96:0/492940849 learned_addr learned my addr 172.21.15.96:0/492940849 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696589700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe69006b0f0 0x7fe6901993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe695d88700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690108a30 0x7fe690199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 -- 172.21.15.96:0/492940849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe69006b0f0 msgr2=0x7fe6901993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe69006b0f0 0x7fe6901993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 -- 172.21.15.96:0/492940849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690108a30 msgr2=0x7fe690199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690108a30 0x7fe690199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 -- 172.21.15.96:0/492940849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69019e7e0 con 0x7fe69019dcc0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe696d8a700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 0x7fe69019e140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe68800ada0 tx=0x7fe68800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688017020 con 0x7fe69019dcc0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68800cd80 con 0x7fe69019dcc0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69019ead0 con 0x7fe69019dcc0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688005260 con 0x7fe69019dcc0 2024-04-25T20:54:19.208 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.204+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe690101c80 con 0x7fe69019dcc0 2024-04-25T20:54:19.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.205+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe690104170 con 0x7fe69019dcc0 2024-04-25T20:54:19.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.207+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe68800a090 con 0x7fe69019dcc0 2024-04-25T20:54:19.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.207+0000 7fe6877fe700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fe67c0a3c40 0x7fe67c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:19.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.207+0000 7fe696589700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fe67c0a3c40 0x7fe67c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:19.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.207+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fe6880cfc60 con 0x7fe69019dcc0 2024-04-25T20:54:19.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.208+0000 7fe696589700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fe67c0a3c40 0x7fe67c0a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005950 tx=0x7fe68c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:19.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.211+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe688021020 con 0x7fe69019dcc0 2024-04-25T20:54:19.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.488+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe69002df80 con 0x7fe69019dcc0 2024-04-25T20:54:19.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.488+0000 7fe6877fe700 1 -- 172.21.15.96:0/492940849 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10470 (secure 0 0 0) 0x7fe68809cb70 con 0x7fe69019dcc0 2024-04-25T20:54:19.490 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:19.490 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24443,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.490+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fe67c0a3c40 msgr2=0x7fe67c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.490+0000 7fe6987ed700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fe67c0a3c40 0x7fe67c0a6100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe68c005950 tx=0x7fe68c00a300 comp rx=0 tx=0).stop 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 msgr2=0x7fe69019e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 0x7fe69019e140 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe68800ada0 tx=0x7fe68800c330 comp rx=0 tx=0).stop 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 shutdown_connections 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fe67c0a3c40 0x7fe67c0a6100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe69019dcc0 0x7fe69019e140 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe69006b0f0 0x7fe6901993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 --2- 172.21.15.96:0/492940849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe690108a30 0x7fe690199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:19.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 >> 172.21.15.96:0/492940849 conn(0x7fe6900788f0 msgr2=0x7fe6900ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:19.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 shutdown_connections 2024-04-25T20:54:19.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:19.491+0000 7fe6987ed700 1 -- 172.21.15.96:0/492940849 wait complete. 2024-04-25T20:54:19.493 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 13 2024-04-25T20:54:20.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-25T20:54:20.415 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:20 smithi096 ceph-mon[29288]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:20 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/492940849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:20 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:20 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:20 smithi097 ceph-mon[32667]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:20 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/492940849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:20 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:20 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:20 smithi139 ceph-mon[33027]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:20 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/492940849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:54:20.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:20 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:20.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:20 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:21.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:21.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:21.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:21.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:21.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:21.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:54:21.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:54:21.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:54:22.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 -- 172.21.15.96:0/963396862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084106990 msgr2=0x7f4084106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:22.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 --2- 172.21.15.96:0/963396862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084106990 0x7f4084106d70 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f407c00b0a0 tx=0x7f407c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 -- 172.21.15.96:0/963396862 shutdown_connections 2024-04-25T20:54:22.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 --2- 172.21.15.96:0/963396862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f40841026e0 0x7f408410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 --2- 172.21.15.96:0/963396862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4084106990 0x7f4084106d70 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 --2- 172.21.15.96:0/963396862 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4084101d20 0x7f40841021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 -- 172.21.15.96:0/963396862 >> 172.21.15.96:0/963396862 conn(0x7f40840788f0 msgr2=0x7f4084078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 -- 172.21.15.96:0/963396862 shutdown_connections 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.058+0000 7f408964d700 1 -- 172.21.15.96:0/963396862 wait complete. 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 Processor -- start 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 -- start start 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d20 0x7f4084199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 0x7f40841996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4084106990 0x7f408419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4084111b40 con 0x7f40841026e0 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f40841119c0 con 0x7f4084101d20 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f408964d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4084111cc0 con 0x7f4084106990 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 0x7f40841996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:22.062 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 0x7f40841996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53942/0 (socket says 172.21.15.96:53942) 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 -- 172.21.15.96:0/516580435 learned_addr learned my addr 172.21.15.96:0/516580435 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f4082ffd700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d20 0x7f4084199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 -- 172.21.15.96:0/516580435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4084106990 msgr2=0x7f408419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4084106990 0x7f408419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 -- 172.21.15.96:0/516580435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d20 msgr2=0x7f4084199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.059+0000 7f40827fc700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d20 0x7f4084199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f40827fc700 1 -- 172.21.15.96:0/516580435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f408419e190 con 0x7f40841026e0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f40827fc700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 0x7f40841996c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f407800c920 tx=0x7f407800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078007b00 con 0x7f40841026e0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4078004ce0 con 0x7f40841026e0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f408419e480 con 0x7f40841026e0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4078005650 con 0x7f40841026e0 2024-04-25T20:54:22.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.060+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4084103510 con 0x7f40841026e0 2024-04-25T20:54:22.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.061+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4084103080 con 0x7f40841026e0 2024-04-25T20:54:22.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.062+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4078004750 con 0x7f40841026e0 2024-04-25T20:54:22.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.063+0000 7f406bfff700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f406c0a3c40 0x7f406c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:22.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.063+0000 7f4082ffd700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f406c0a3c40 0x7f406c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:22.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.063+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7f40780cfec0 con 0x7f40841026e0 2024-04-25T20:54:22.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.063+0000 7f4082ffd700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f406c0a3c40 0x7f406c0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f407c006130 tx=0x7f407c006080 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:22.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.067+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4078014030 con 0x7f40841026e0 2024-04-25T20:54:22.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.269+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f408402df80 con 0x7f40841026e0 2024-04-25T20:54:22.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.269+0000 7f406bfff700 1 -- 172.21.15.96:0/516580435 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v84) ==== 84+0+65 (secure 0 0 0) 0x7f4078098020 con 0x7f40841026e0 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout:[ 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: "poolnum": 1, 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: "poolname": ".mgr" 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:54:22.271 INFO:teuthology.orchestra.run.smithi096.stdout:] 2024-04-25T20:54:22.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f406c0a3c40 msgr2=0x7f406c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:22.273 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f406c0a3c40 0x7f406c0a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f407c006130 tx=0x7f407c006080 comp rx=0 tx=0).stop 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 msgr2=0x7f40841996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 0x7f40841996c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f407800c920 tx=0x7f407800cce0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 shutdown_connections 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f406c0a3c40 0x7f406c0a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4084106990 0x7f408419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4084101d20 0x7f4084199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 --2- 172.21.15.96:0/516580435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40841026e0 0x7f40841996c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 >> 172.21.15.96:0/516580435 conn(0x7f40840788f0 msgr2=0x7f408406b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 shutdown_connections 2024-04-25T20:54:22.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:22.272+0000 7f408964d700 1 -- 172.21.15.96:0/516580435 wait complete. 2024-04-25T20:54:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:22 smithi097 ceph-mon[32667]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:22 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/516580435' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T20:54:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:22 smithi139 ceph-mon[33027]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:22.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:22 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/516580435' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T20:54:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:22 smithi096 ceph-mon[29288]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:22 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/516580435' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-25T20:54:22.873 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T20:54:22.873 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T20:54:23.119 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:24.460 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:24 smithi096 ceph-mon[29288]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:24 smithi097 ceph-mon[32667]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:24.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:24 smithi139 ceph-mon[33027]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:24.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 -- 172.21.15.96:0/1081534218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908102a90 msgr2=0x7fa908102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:24.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 --2- 172.21.15.96:0/1081534218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908102a90 0x7fa908102f10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa904009990 tx=0x7fa904009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:24.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 -- 172.21.15.96:0/1081534218 shutdown_connections 2024-04-25T20:54:24.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 --2- 172.21.15.96:0/1081534218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa908103450 0x7fa90810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 --2- 172.21.15.96:0/1081534218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908108a50 0x7fa908108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 --2- 172.21.15.96:0/1081534218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908102a90 0x7fa908102f10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.683+0000 7fa90f345700 1 -- 172.21.15.96:0/1081534218 >> 172.21.15.96:0/1081534218 conn(0x7fa908079ba0 msgr2=0x7fa908079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 -- 172.21.15.96:0/1081534218 shutdown_connections 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 -- 172.21.15.96:0/1081534218 wait complete. 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 Processor -- start 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 -- start start 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908102a90 0x7fa908074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa908103450 0x7fa908075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 0x7fa9080757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa908112c60 con 0x7fa908108a50 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa908112ae0 con 0x7fa908102a90 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90f345700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa908112de0 con 0x7fa908103450 2024-04-25T20:54:24.687 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90d8e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 0x7fa9080757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90d8e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 0x7fa9080757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53948/0 (socket says 172.21.15.96:53948) 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.684+0000 7fa90d8e2700 1 -- 172.21.15.96:0/3782191417 learned_addr learned my addr 172.21.15.96:0/3782191417 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90c8e0700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa908103450 0x7fa908075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d0e1700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908102a90 0x7fa908074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d8e2700 1 -- 172.21.15.96:0/3782191417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa908103450 msgr2=0x7fa908075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d8e2700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa908103450 0x7fa908075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d8e2700 1 -- 172.21.15.96:0/3782191417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908102a90 msgr2=0x7fa908074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d8e2700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908102a90 0x7fa908074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d8e2700 1 -- 172.21.15.96:0/3782191417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9081a7720 con 0x7fa908108a50 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d0e1700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908102a90 0x7fa908074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90d8e2700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 0x7fa9080757c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa90000ada0 tx=0x7fa90000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900017020 con 0x7fa908108a50 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9081a7a10 con 0x7fa908108a50 2024-04-25T20:54:24.688 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa90000ce30 con 0x7fa908108a50 2024-04-25T20:54:24.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9000054a0 con 0x7fa908108a50 2024-04-25T20:54:24.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.685+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9081a7f00 con 0x7fa908108a50 2024-04-25T20:54:24.689 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.686+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa908069730 con 0x7fa908108a50 2024-04-25T20:54:24.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.687+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa90000c980 con 0x7fa908108a50 2024-04-25T20:54:24.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.688+0000 7fa8fe7fc700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8f40a3c40 0x7fa8f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:24.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.688+0000 7fa90d0e1700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8f40a3c40 0x7fa8f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:24.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.688+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7127+0+0 (secure 0 0 0) 0x7fa9000cfd10 con 0x7fa908108a50 2024-04-25T20:54:24.690 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.688+0000 7fa90d0e1700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8f40a3c40 0x7fa8f40a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007900 tx=0x7fa8f8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:24.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.691+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa900021020 con 0x7fa908108a50 2024-04-25T20:54:24.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:24.892+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa90802df80 con 0x7fa908108a50 2024-04-25T20:54:25.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.344+0000 7fa8fe7fc700 1 -- 172.21.15.96:0/3782191417 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v85) ==== 138+0+0 (secure 0 0 0) 0x7fa90009cc20 con 0x7fa908108a50 2024-04-25T20:54:25.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8f40a3c40 msgr2=0x7fa8f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8f40a3c40 0x7fa8f40a6100 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa8f8007900 tx=0x7fa8f8008040 comp rx=0 tx=0).stop 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 msgr2=0x7fa9080757c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 0x7fa9080757c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa90000ada0 tx=0x7fa90000c330 comp rx=0 tx=0).stop 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 shutdown_connections 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8f40a3c40 0x7fa8f40a6100 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa908108a50 0x7fa9080757c0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa908102a90 0x7fa908074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 --2- 172.21.15.96:0/3782191417 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa908103450 0x7fa908075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.346+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 >> 172.21.15.96:0/3782191417 conn(0x7fa908079ba0 msgr2=0x7fa9081017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.347+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 shutdown_connections 2024-04-25T20:54:25.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:25.347+0000 7fa90f345700 1 -- 172.21.15.96:0/3782191417 wait complete. 2024-04-25T20:54:25.349 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' created 2024-04-25T20:54:25.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:25 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3782191417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T20:54:25.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:25 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3782191417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T20:54:25.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:25 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3782191417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-25T20:54:26.021 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T20:54:26.268 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:26 smithi097 ceph-mon[32667]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:26.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:26 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3782191417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T20:54:26.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:26 smithi097 ceph-mon[32667]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T20:54:26.617 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:26 smithi096 ceph-mon[29288]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:26.617 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3782191417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T20:54:26.617 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:26 smithi096 ceph-mon[29288]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T20:54:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:26 smithi139 ceph-mon[33027]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:26 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3782191417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-25T20:54:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:26 smithi139 ceph-mon[33027]: osdmap e85: 12 total, 12 up, 12 in 2024-04-25T20:54:27.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:27 smithi097 ceph-mon[32667]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T20:54:27.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:27 smithi139 ceph-mon[33027]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T20:54:27.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:27 smithi096 ceph-mon[29288]: osdmap e86: 12 total, 12 up, 12 in 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 -- 172.21.15.96:0/1091170391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818106500 msgr2=0x7f581810e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 --2- 172.21.15.96:0/1091170391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818106500 0x7f581810e040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f581000b0a0 tx=0x7f581000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 -- 172.21.15.96:0/1091170391 shutdown_connections 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 --2- 172.21.15.96:0/1091170391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818106500 0x7f581810e040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 --2- 172.21.15.96:0/1091170391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818101660 0x7f5818101a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 --2- 172.21.15.96:0/1091170391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818102010 0x7f5818105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 -- 172.21.15.96:0/1091170391 >> 172.21.15.96:0/1091170391 conn(0x7f5818076c60 msgr2=0x7f5818077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 -- 172.21.15.96:0/1091170391 shutdown_connections 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 -- 172.21.15.96:0/1091170391 wait complete. 2024-04-25T20:54:28.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 Processor -- start 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.044+0000 7f581d42c700 1 -- start start 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 0x7f5818198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818102010 0x7f58181994c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818106500 0x7f581819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5818110ab0 con 0x7f5818101660 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5818110930 con 0x7f5818102010 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5818110c30 con 0x7f5818106500 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 0x7f5818198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 0x7f5818198f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41916/0 (socket says 172.21.15.96:41916) 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 -- 172.21.15.96:0/63685478 learned_addr learned my addr 172.21.15.96:0/63685478 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f58177fe700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818106500 0x7f581819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 -- 172.21.15.96:0/63685478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818106500 msgr2=0x7f581819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818106500 0x7f581819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 -- 172.21.15.96:0/63685478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818102010 msgr2=0x7f58181994c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T20:54:28.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818102010 0x7f58181994c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 -- 172.21.15.96:0/63685478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581819df90 con 0x7f5818101660 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f58177fe700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818106500 0x7f581819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f5816ffd700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 0x7f5818198f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c9b0 tx=0x7f580c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c007b00 con 0x7f5818101660 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581819e280 con 0x7f5818101660 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c007ca0 con 0x7f5818101660 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.045+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c0187c0 con 0x7f5818101660 2024-04-25T20:54:28.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.046+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58181a5b00 con 0x7f5818101660 2024-04-25T20:54:28.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.046+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5818069730 con 0x7f5818101660 2024-04-25T20:54:28.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.048+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f580c00fa20 con 0x7f5818101660 2024-04-25T20:54:28.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.048+0000 7f57fffff700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f58000a3c30 0x7f58000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:28.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.048+0000 7f58167fc700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f58000a3c30 0x7f58000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:28.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.048+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7497+0+0 (secure 0 0 0) 0x7f580c0d03d0 con 0x7f5818101660 2024-04-25T20:54:28.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.049+0000 7f58167fc700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f58000a3c30 0x7f58000a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5818101140 tx=0x7f5808005c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:28.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.052+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f580c014030 con 0x7f5818101660 2024-04-25T20:54:28.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.256+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f581802df80 con 0x7f5818101660 2024-04-25T20:54:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.371+0000 7f57fffff700 1 -- 172.21.15.96:0/63685478 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v88) ==== 170+0+0 (secure 0 0 0) 0x7f580c09d170 con 0x7f5818101660 2024-04-25T20:54:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f58000a3c30 msgr2=0x7f58000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f58000a3c30 0x7f58000a60f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5818101140 tx=0x7f5808005c10 comp rx=0 tx=0).stop 2024-04-25T20:54:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 msgr2=0x7f5818198f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 0x7f5818198f80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f580c00c9b0 tx=0x7f580c00cd70 comp rx=0 tx=0).stop 2024-04-25T20:54:28.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 shutdown_connections 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f58000a3c30 0x7f58000a60f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5818106500 0x7f581819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5818101660 0x7f5818198f80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 --2- 172.21.15.96:0/63685478 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5818102010 0x7f58181994c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.373+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 >> 172.21.15.96:0/63685478 conn(0x7f5818076c60 msgr2=0x7f58180ff9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.374+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 shutdown_connections 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:28.374+0000 7f581d42c700 1 -- 172.21.15.96:0/63685478 wait complete. 2024-04-25T20:54:28.396 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' created 2024-04-25T20:54:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:28 smithi139 ceph-mon[33027]: pgmap v295: 33 pgs: 2 creating+activating, 11 creating+peering, 15 unknown, 5 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:28.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:28 smithi139 ceph-mon[33027]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T20:54:28.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:28 smithi139 ceph-mon[33027]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T20:54:28.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:28 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/63685478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T20:54:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:28 smithi096 ceph-mon[29288]: pgmap v295: 33 pgs: 2 creating+activating, 11 creating+peering, 15 unknown, 5 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:28 smithi096 ceph-mon[29288]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T20:54:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:28 smithi096 ceph-mon[29288]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T20:54:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:28 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/63685478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T20:54:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:28 smithi097 ceph-mon[32667]: pgmap v295: 33 pgs: 2 creating+activating, 11 creating+peering, 15 unknown, 5 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:28.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:28 smithi097 ceph-mon[32667]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-25T20:54:28.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:28 smithi097 ceph-mon[32667]: osdmap e87: 12 total, 12 up, 12 in 2024-04-25T20:54:28.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:28 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/63685478' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-25T20:54:29.028 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T20:54:29.270 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:29.382 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:29 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/63685478' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T20:54:29.382 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:29 smithi096 ceph-mon[29288]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T20:54:29.382 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:29 smithi096 ceph-mon[29288]: pgmap v298: 97 pgs: 2 creating+activating, 18 creating+peering, 62 unknown, 15 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:29.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:29 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/63685478' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T20:54:29.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:29 smithi097 ceph-mon[32667]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T20:54:29.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:29 smithi097 ceph-mon[32667]: pgmap v298: 97 pgs: 2 creating+activating, 18 creating+peering, 62 unknown, 15 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:29.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:29 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/63685478' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-25T20:54:29.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:29 smithi139 ceph-mon[33027]: osdmap e88: 12 total, 12 up, 12 in 2024-04-25T20:54:29.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:29 smithi139 ceph-mon[33027]: pgmap v298: 97 pgs: 2 creating+activating, 18 creating+peering, 62 unknown, 15 active+clean; 577 KiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:30.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:30 smithi096 ceph-mon[29288]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T20:54:30.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:30 smithi097 ceph-mon[32667]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T20:54:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:30 smithi139 ceph-mon[33027]: osdmap e89: 12 total, 12 up, 12 in 2024-04-25T20:54:30.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 -- 172.21.15.96:0/1071104487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9006a750 msgr2=0x7fbc9006abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:30.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 --2- 172.21.15.96:0/1071104487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9006a750 0x7fbc9006abb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009990 tx=0x7fbc8c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 -- 172.21.15.96:0/1071104487 shutdown_connections 2024-04-25T20:54:30.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 --2- 172.21.15.96:0/1071104487 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 0x7fbc9010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 --2- 172.21.15.96:0/1071104487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc90108a30 0x7fbc90108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 --2- 172.21.15.96:0/1071104487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9006a750 0x7fbc9006abb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 -- 172.21.15.96:0/1071104487 >> 172.21.15.96:0/1071104487 conn(0x7fbc900788f0 msgr2=0x7fbc90078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 -- 172.21.15.96:0/1071104487 shutdown_connections 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.971+0000 7fbc98045700 1 -- 172.21.15.96:0/1071104487 wait complete. 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 Processor -- start 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 -- start start 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9006a750 0x7fbc9019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 0x7fbc9019b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 0x7fbc9019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901026e0 con 0x7fbc90108a30 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90102560 con 0x7fbc9006a750 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc98045700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90102860 con 0x7fbc9006b0f0 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc965e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 0x7fbc9019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:30.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc965e2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 0x7fbc9019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41934/0 (socket says 172.21.15.96:41934) 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.972+0000 7fbc965e2700 1 -- 172.21.15.96:0/437513032 learned_addr learned my addr 172.21.15.96:0/437513032 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc95de1700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9006a750 0x7fbc9019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc955e0700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 0x7fbc9019b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc965e2700 1 -- 172.21.15.96:0/437513032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 msgr2=0x7fbc9019b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc965e2700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 0x7fbc9019b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc965e2700 1 -- 172.21.15.96:0/437513032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9006a750 msgr2=0x7fbc9019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc965e2700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9006a750 0x7fbc9019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc965e2700 1 -- 172.21.15.96:0/437513032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc901a0320 con 0x7fbc90108a30 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc955e0700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 0x7fbc9019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc965e2700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 0x7fbc9019fbe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b970 tx=0x7fbc8800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88019070 con 0x7fbc90108a30 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc901a0610 con 0x7fbc90108a30 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc8800cde0 con 0x7fbc90108a30 2024-04-25T20:54:30.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88007370 con 0x7fbc90108a30 2024-04-25T20:54:30.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.973+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901a7e90 con 0x7fbc90108a30 2024-04-25T20:54:30.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.974+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc90069730 con 0x7fbc90108a30 2024-04-25T20:54:30.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.975+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbc8801c070 con 0x7fbc90108a30 2024-04-25T20:54:30.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.976+0000 7fbc86ffd700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbc7c0a3c40 0x7fbc7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:30.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.976+0000 7fbc95de1700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbc7c0a3c40 0x7fbc7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:30.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.976+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 7879+0+0 (secure 0 0 0) 0x7fbc880cfe10 con 0x7fbc90108a30 2024-04-25T20:54:30.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.976+0000 7fbc95de1700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbc7c0a3c40 0x7fbc7c0a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc80005950 tx=0x7fbc8000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:30.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:30.980+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc88023070 con 0x7fbc90108a30 2024-04-25T20:54:31.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.231+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fbc901a80c0 con 0x7fbc90108a30 2024-04-25T20:54:31.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.392+0000 7fbc86ffd700 1 -- 172.21.15.96:0/437513032 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fbc8809c9a0 con 0x7fbc90108a30 2024-04-25T20:54:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbc7c0a3c40 msgr2=0x7fbc7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:31.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbc7c0a3c40 0x7fbc7c0a6100 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fbc80005950 tx=0x7fbc8000a300 comp rx=0 tx=0).stop 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 msgr2=0x7fbc9019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 0x7fbc9019fbe0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b970 tx=0x7fbc8800bc80 comp rx=0 tx=0).stop 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 shutdown_connections 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fbc7c0a3c40 0x7fbc7c0a6100 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc90108a30 0x7fbc9019fbe0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbc9006a750 0x7fbc9019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 --2- 172.21.15.96:0/437513032 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbc9006b0f0 0x7fbc9019b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 >> 172.21.15.96:0/437513032 conn(0x7fbc900788f0 msgr2=0x7fbc901010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 shutdown_connections 2024-04-25T20:54:31.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:31.394+0000 7fbc98045700 1 -- 172.21.15.96:0/437513032 wait complete. 2024-04-25T20:54:31.397 INFO:teuthology.orchestra.run.smithi096.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T20:54:31.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:31 smithi096 ceph-mon[29288]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T20:54:31.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:31 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/437513032' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T20:54:31.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:31 smithi096 ceph-mon[29288]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T20:54:31.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:31 smithi096 ceph-mon[29288]: pgmap v301: 97 pgs: 2 creating+activating, 10 creating+peering, 34 unknown, 51 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:31.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:31 smithi097 ceph-mon[32667]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T20:54:31.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:31 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/437513032' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T20:54:31.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:31 smithi097 ceph-mon[32667]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T20:54:31.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:31 smithi097 ceph-mon[32667]: pgmap v301: 97 pgs: 2 creating+activating, 10 creating+peering, 34 unknown, 51 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:31.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:31 smithi139 ceph-mon[33027]: osdmap e90: 12 total, 12 up, 12 in 2024-04-25T20:54:31.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:31 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/437513032' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-25T20:54:31.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:31 smithi139 ceph-mon[33027]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-25T20:54:31.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:31 smithi139 ceph-mon[33027]: pgmap v301: 97 pgs: 2 creating+activating, 10 creating+peering, 34 unknown, 51 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:54:32.045 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:54:32.285 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:32.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:32 smithi096 ceph-mon[29288]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T20:54:32.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:32 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/437513032' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T20:54:32.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:32 smithi096 ceph-mon[29288]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T20:54:32.669 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:32 smithi096 ceph-mon[29288]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T20:54:32.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:32 smithi097 ceph-mon[32667]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T20:54:32.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:32 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/437513032' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T20:54:32.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:32 smithi097 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T20:54:32.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:32 smithi097 ceph-mon[32667]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T20:54:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:32 smithi139 ceph-mon[33027]: osdmap e91: 12 total, 12 up, 12 in 2024-04-25T20:54:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:32 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/437513032' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-25T20:54:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:32 smithi139 ceph-mon[33027]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-25T20:54:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:32 smithi139 ceph-mon[33027]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-25T20:54:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:33 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] up:active 2024-04-25T20:54:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:33 smithi139 ceph-mon[33027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:33 smithi139 ceph-mon[33027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:33 smithi139 ceph-mon[33027]: pgmap v303: 97 pgs: 7 creating+peering, 12 unknown, 78 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 421 B/s wr, 5 op/s 2024-04-25T20:54:33.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:33 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] up:active 2024-04-25T20:54:33.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:33 smithi096 ceph-mon[29288]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:33.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:33 smithi096 ceph-mon[29288]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:33.726 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:33 smithi096 ceph-mon[29288]: pgmap v303: 97 pgs: 7 creating+peering, 12 unknown, 78 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 421 B/s wr, 5 op/s 2024-04-25T20:54:33.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:33 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] up:active 2024-04-25T20:54:33.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:33 smithi097 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:33.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:33 smithi097 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:33.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:33 smithi097 ceph-mon[32667]: pgmap v303: 97 pgs: 7 creating+peering, 12 unknown, 78 active+clean; 577 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 421 B/s wr, 5 op/s 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 -- 172.21.15.96:0/170953314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 msgr2=0x7fea2010e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 --2- 172.21.15.96:0/170953314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2010e810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fea1800b0a0 tx=0x7fea1800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 -- 172.21.15.96:0/170953314 shutdown_connections 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 --2- 172.21.15.96:0/170953314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2010e810 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 --2- 172.21.15.96:0/170953314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea20107940 0x7fea20107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 --2- 172.21.15.96:0/170953314 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea201019d0 0x7fea20101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 -- 172.21.15.96:0/170953314 >> 172.21.15.96:0/170953314 conn(0x7fea20077990 msgr2=0x7fea20077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 -- 172.21.15.96:0/170953314 shutdown_connections 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.093+0000 7fea26084700 1 -- 172.21.15.96:0/170953314 wait complete. 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 Processor -- start 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 -- start start 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea201019d0 0x7fea20199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea20107940 0x7fea201a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea20111b20 con 0x7fea20102390 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fea201119a0 con 0x7fea201019d0 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea26084700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fea20111ca0 con 0x7fea20107940 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea1effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea1effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41950/0 (socket says 172.21.15.96:41950) 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.094+0000 7fea1effd700 1 -- 172.21.15.96:0/176973054 learned_addr learned my addr 172.21.15.96:0/176973054 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:34.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1f7fe700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea201019d0 0x7fea20199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1ffff700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea20107940 0x7fea201a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1effd700 1 -- 172.21.15.96:0/176973054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea20107940 msgr2=0x7fea201a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1effd700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea20107940 0x7fea201a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1effd700 1 -- 172.21.15.96:0/176973054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea201019d0 msgr2=0x7fea20199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1effd700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea201019d0 0x7fea20199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1effd700 1 -- 172.21.15.96:0/176973054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea201a53c0 con 0x7fea20102390 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1effd700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2019a340 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea1400d950 tx=0x7fea1400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14016070 con 0x7fea20102390 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea14010460 con 0x7fea20102390 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1400f620 con 0x7fea20102390 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea201a55f0 con 0x7fea20102390 2024-04-25T20:54:34.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.095+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea201a5b80 con 0x7fea20102390 2024-04-25T20:54:34.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.096+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea20069730 con 0x7fea20102390 2024-04-25T20:54:34.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.097+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea14019070 con 0x7fea20102390 2024-04-25T20:54:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.098+0000 7fea1cff9700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fea080a3cc0 0x7fea080a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.098+0000 7fea1f7fe700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fea080a3cc0 0x7fea080a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.098+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7fea140cfe80 con 0x7fea20102390 2024-04-25T20:54:34.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.098+0000 7fea1f7fe700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fea080a3cc0 0x7fea080a6180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea10005fd0 tx=0x7fea1000af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:34.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.102+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea1409c9b0 con 0x7fea20102390 2024-04-25T20:54:34.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.301+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fea201a5de0 con 0x7fea20102390 2024-04-25T20:54:34.303 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.303+0000 7fea1cff9700 1 -- 172.21.15.96:0/176973054 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22184 (secure 0 0 0) 0x7fea14096e20 con 0x7fea20102390 2024-04-25T20:54:34.304 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:34.304 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":91,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:54:31.232543+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fea080a3cc0 msgr2=0x7fea080a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fea080a3cc0 0x7fea080a6180 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fea10005fd0 tx=0x7fea1000af20 comp rx=0 tx=0).stop 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 msgr2=0x7fea2019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2019a340 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea1400d950 tx=0x7fea1400dd10 comp rx=0 tx=0).stop 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 shutdown_connections 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fea080a3cc0 0x7fea080a6180 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fea20107940 0x7fea201a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fea201019d0 0x7fea20199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 --2- 172.21.15.96:0/176973054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea20102390 0x7fea2019a340 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:34.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 >> 172.21.15.96:0/176973054 conn(0x7fea20077990 msgr2=0x7fea200ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:34.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 shutdown_connections 2024-04-25T20:54:34.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:34.305+0000 7fea26084700 1 -- 172.21.15.96:0/176973054 wait complete. 2024-04-25T20:54:34.605 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:34 smithi096 ceph-mon[29288]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T20:54:34.605 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:34 smithi096 ceph-mon[29288]: Cluster is now healthy 2024-04-25T20:54:34.605 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:34 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/176973054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:54:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:34 smithi097 ceph-mon[32667]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T20:54:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:34 smithi097 ceph-mon[32667]: Cluster is now healthy 2024-04-25T20:54:34.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:34 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/176973054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:54:34.614 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 20:54:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: 2024-04-25T20:54:34.299+0000 7ff7be069700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T20:54:34.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:34 smithi139 ceph-mon[33027]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-25T20:54:34.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:34 smithi139 ceph-mon[33027]: Cluster is now healthy 2024-04-25T20:54:34.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:34 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/176973054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:54:34.951 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:54:35.190 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:35.563 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:35 smithi096 ceph-mon[29288]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-04-25T20:54:35.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:35 smithi097 ceph-mon[32667]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-04-25T20:54:35.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:35 smithi139 ceph-mon[33027]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s wr, 5 op/s 2024-04-25T20:54:36.968 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.965+0000 7ffaff0dd700 1 -- 172.21.15.96:0/2542139052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8102330 msgr2=0x7ffaf810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.965+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/2542139052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8102330 0x7ffaf810e7f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffaf000b0a0 tx=0x7ffaf000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- 172.21.15.96:0/2542139052 shutdown_connections 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/2542139052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8102330 0x7ffaf810e7f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/2542139052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaf8107920 0x7ffaf8107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/2542139052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaf8101970 0x7ffaf8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- 172.21.15.96:0/2542139052 >> 172.21.15.96:0/2542139052 conn(0x7ffaf8078910 msgr2=0x7ffaf8078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- 172.21.15.96:0/2542139052 shutdown_connections 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- 172.21.15.96:0/2542139052 wait complete. 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 Processor -- start 2024-04-25T20:54:36.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- start start 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 0x7ffaf819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaf8102330 0x7ffaf819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaf8107920 0x7ffaf819fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8111b60 con 0x7ffaf8101970 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf81119e0 con 0x7ffaf8102330 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffaff0dd700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffaf8111ce0 con 0x7ffaf8107920 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.966+0000 7ffafce79700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 0x7ffaf819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 0x7ffaf819b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35344/0 (socket says 172.21.15.96:35344) 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 -- 172.21.15.96:0/3994582087 learned_addr learned my addr 172.21.15.96:0/3994582087 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafd67a700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaf8107920 0x7ffaf819fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffaf7fff700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaf8102330 0x7ffaf819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 -- 172.21.15.96:0/3994582087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaf8107920 msgr2=0x7ffaf819fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:36.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaf8107920 0x7ffaf819fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 -- 172.21.15.96:0/3994582087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaf8102330 msgr2=0x7ffaf819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaf8102330 0x7ffaf819b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 -- 172.21.15.96:0/3994582087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaf81a02a0 con 0x7ffaf8101970 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffafce79700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 0x7ffaf819b290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c920 tx=0x7ffaec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec007b00 con 0x7ffaf8101970 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaf81a0590 con 0x7ffaf8101970 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffaec004ce0 con 0x7ffaf8101970 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaec005500 con 0x7ffaf8101970 2024-04-25T20:54:36.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.967+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaf81a7e10 con 0x7ffaf8101970 2024-04-25T20:54:36.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.968+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffaf8069730 con 0x7ffaf8101970 2024-04-25T20:54:36.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.969+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffaec007ca0 con 0x7ffaf8101970 2024-04-25T20:54:36.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.970+0000 7ffaf5ffb700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffae80a3c40 0x7ffae80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:36.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.970+0000 7ffaf7fff700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffae80a3c40 0x7ffae80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:36.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.970+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7ffaec0cfe60 con 0x7ffaf8101970 2024-04-25T20:54:36.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.970+0000 7ffaf7fff700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffae80a3c40 0x7ffae80a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffae4006fd0 tx=0x7ffae4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:36.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:36.975+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffaec014030 con 0x7ffaf8101970 2024-04-25T20:54:37.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.173+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffaf819c5f0 con 0x7ffaf8101970 2024-04-25T20:54:37.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.174+0000 7ffaf5ffb700 1 -- 172.21.15.96:0/3994582087 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22184 (secure 0 0 0) 0x7ffaec09c990 con 0x7ffaf8101970 2024-04-25T20:54:37.176 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:54:37.176 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":91,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:54:31.232543+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffae80a3c40 msgr2=0x7ffae80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffae80a3c40 0x7ffae80a6100 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ffae4006fd0 tx=0x7ffae4008040 comp rx=0 tx=0).stop 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 msgr2=0x7ffaf819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 0x7ffaf819b290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffaec00c920 tx=0x7ffaec00cc30 comp rx=0 tx=0).stop 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 shutdown_connections 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffae80a3c40 0x7ffae80a6100 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffaf8107920 0x7ffaf819fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaf8101970 0x7ffaf819b290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 --2- 172.21.15.96:0/3994582087 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffaf8102330 0x7ffaf819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.176+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 >> 172.21.15.96:0/3994582087 conn(0x7ffaf8078910 msgr2=0x7ffaf810ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:37.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.177+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 shutdown_connections 2024-04-25T20:54:37.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:37.177+0000 7ffaff0dd700 1 -- 172.21.15.96:0/3994582087 wait complete. 2024-04-25T20:54:37.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:37 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3994582087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:54:37.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:37 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3994582087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:54:37.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:37 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3994582087' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:54:37.866 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-25T20:54:38.103 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:38 smithi096 ceph-mon[29288]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-25T20:54:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:38 smithi097 ceph-mon[32667]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-25T20:54:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:38 smithi139 ceph-mon[33027]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-04-25T20:54:39.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 -- 172.21.15.96:0/1087030581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406b1e0 msgr2=0x7faf7406b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:39.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/1087030581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406b1e0 0x7faf7406b660 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7faf680099c0 tx=0x7faf68009cd0 comp rx=0 tx=0).stop 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 -- 172.21.15.96:0/1087030581 shutdown_connections 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/1087030581 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf7406bba0 0x7faf7410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/1087030581 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf74108a20 0x7faf74108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/1087030581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406b1e0 0x7faf7406b660 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 -- 172.21.15.96:0/1087030581 >> 172.21.15.96:0/1087030581 conn(0x7faf740779b0 msgr2=0x7faf74077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 -- 172.21.15.96:0/1087030581 shutdown_connections 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.932+0000 7faf7a5b1700 1 -- 172.21.15.96:0/1087030581 wait complete. 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 Processor -- start 2024-04-25T20:54:39.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 -- start start 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf7406b1e0 0x7faf741991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 0x7faf741996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74108a20 0x7faf7419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf74112c70 con 0x7faf7406bba0 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7faf74112af0 con 0x7faf74108a20 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf7a5b1700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7faf74112df0 con 0x7faf7406b1e0 2024-04-25T20:54:39.936 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 0x7faf741996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 0x7faf741996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35364/0 (socket says 172.21.15.96:35364) 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 -- 172.21.15.96:0/2198193336 learned_addr learned my addr 172.21.15.96:0/2198193336 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf78b4e700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74108a20 0x7faf7419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf73fff700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf7406b1e0 0x7faf741991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 -- 172.21.15.96:0/2198193336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf7406b1e0 msgr2=0x7faf741991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf7406b1e0 0x7faf741991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 -- 172.21.15.96:0/2198193336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74108a20 msgr2=0x7faf7419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74108a20 0x7faf7419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.933+0000 7faf737fe700 1 -- 172.21.15.96:0/2198193336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf7419e160 con 0x7faf7406bba0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.934+0000 7faf737fe700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 0x7faf741996e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faf6800b580 tx=0x7faf680047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.934+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf68027070 con 0x7faf7406bba0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.934+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7419e3f0 con 0x7faf7406bba0 2024-04-25T20:54:39.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.934+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf680040a0 con 0x7faf7406bba0 2024-04-25T20:54:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.934+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf6800f460 con 0x7faf7406bba0 2024-04-25T20:54:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.934+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf74101970 con 0x7faf7406bba0 2024-04-25T20:54:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.935+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf741041a0 con 0x7faf7406bba0 2024-04-25T20:54:39.938 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.936+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faf6802a070 con 0x7faf7406bba0 2024-04-25T20:54:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.936+0000 7faf717fa700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faf5c0a3c40 0x7faf5c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.937+0000 7faf73fff700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faf5c0a3c40 0x7faf5c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.937+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7faf680d1730 con 0x7faf7406bba0 2024-04-25T20:54:39.939 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.937+0000 7faf73fff700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faf5c0a3c40 0x7faf5c0a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf64006fd0 tx=0x7faf64008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:39.941 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:39.940+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf68031070 con 0x7faf7406bba0 2024-04-25T20:54:40.185 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.184+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7faf7419a4b0 con 0x7faf7406bba0 2024-04-25T20:54:40.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.290+0000 7faf717fa700 1 -- 172.21.15.96:0/2198193336 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7faf68013070 con 0x7faf7406bba0 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faf5c0a3c40 msgr2=0x7faf5c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faf5c0a3c40 0x7faf5c0a6100 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf64006fd0 tx=0x7faf64008040 comp rx=0 tx=0).stop 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 msgr2=0x7faf741996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 0x7faf741996e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faf6800b580 tx=0x7faf680047b0 comp rx=0 tx=0).stop 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 shutdown_connections 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7faf5c0a3c40 0x7faf5c0a6100 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faf74108a20 0x7faf7419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:40.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7faf7406b1e0 0x7faf741991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:40.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 --2- 172.21.15.96:0/2198193336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf7406bba0 0x7faf741996e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:40.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 >> 172.21.15.96:0/2198193336 conn(0x7faf740779b0 msgr2=0x7faf74100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:40.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 shutdown_connections 2024-04-25T20:54:40.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:40.293+0000 7faf7a5b1700 1 -- 172.21.15.96:0/2198193336 wait complete. 2024-04-25T20:54:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:40 smithi097 ceph-mon[32667]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-25T20:54:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:40 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2198193336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T20:54:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:40 smithi139 ceph-mon[33027]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-25T20:54:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:40 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2198193336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T20:54:40.648 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:40 smithi096 ceph-mon[29288]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 5 op/s 2024-04-25T20:54:40.648 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:40 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2198193336' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-25T20:54:40.967 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-04-25T20:54:40.967 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-04-25T20:54:41.211 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:41.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:41 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2198193336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T20:54:41.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:41 smithi096 ceph-mon[29288]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:41.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:41 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2198193336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T20:54:41.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:41 smithi097 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:41.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:41 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2198193336' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-25T20:54:41.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:41 smithi139 ceph-mon[33027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:42.600 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:42 smithi096 ceph-mon[29288]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T20:54:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:42 smithi097 ceph-mon[32667]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T20:54:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:42 smithi139 ceph-mon[33027]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-25T20:54:42.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 -- 172.21.15.96:0/1570511565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54101970 msgr2=0x7fce54101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:42.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1570511565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54101970 0x7fce54101df0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fce48009990 tx=0x7fce48009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:42.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 -- 172.21.15.96:0/1570511565 shutdown_connections 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1570511565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54102330 0x7fce5410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1570511565 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce54107920 0x7fce54107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1570511565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54101970 0x7fce54101df0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 -- 172.21.15.96:0/1570511565 >> 172.21.15.96:0/1570511565 conn(0x7fce540788f0 msgr2=0x7fce54078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 -- 172.21.15.96:0/1570511565 shutdown_connections 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.842+0000 7fce59ef8700 1 -- 172.21.15.96:0/1570511565 wait complete. 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 Processor -- start 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 -- start start 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54101970 0x7fce54198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce54102330 0x7fce54198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 0x7fce5419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:42.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce54111b00 con 0x7fce54107920 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fce54111980 con 0x7fce54101970 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce59ef8700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fce54111c80 con 0x7fce54102330 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce53fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 0x7fce5419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce53fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 0x7fce5419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35390/0 (socket says 172.21.15.96:35390) 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.843+0000 7fce53fff700 1 -- 172.21.15.96:0/1179438686 learned_addr learned my addr 172.21.15.96:0/1179438686 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce52ffd700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce54102330 0x7fce54198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce537fe700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54101970 0x7fce54198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce53fff700 1 -- 172.21.15.96:0/1179438686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce54102330 msgr2=0x7fce54198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce53fff700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce54102330 0x7fce54198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce53fff700 1 -- 172.21.15.96:0/1179438686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54101970 msgr2=0x7fce54198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce53fff700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54101970 0x7fce54198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce53fff700 1 -- 172.21.15.96:0/1179438686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5419d040 con 0x7fce54107920 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce537fe700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54101970 0x7fce54198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:54:42.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce53fff700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 0x7fce5419c900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007ee0 tx=0x7fce4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:42.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c00dbd0 con 0x7fce54107920 2024-04-25T20:54:42.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5419d330 con 0x7fce54107920 2024-04-25T20:54:42.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fce4c00dd70 con 0x7fce54107920 2024-04-25T20:54:42.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4c005990 con 0x7fce54107920 2024-04-25T20:54:42.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.844+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce541a4bb0 con 0x7fce54107920 2024-04-25T20:54:42.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.845+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce54069730 con 0x7fce54107920 2024-04-25T20:54:42.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.846+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fce4c01a020 con 0x7fce54107920 2024-04-25T20:54:42.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.847+0000 7fce50ff9700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fce3c0a3c40 0x7fce3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:42.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.847+0000 7fce537fe700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fce3c0a3c40 0x7fce3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:42.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.847+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7fce4c0d0100 con 0x7fce54107920 2024-04-25T20:54:42.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.847+0000 7fce537fe700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fce3c0a3c40 0x7fce3c0a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fce44005fd0 tx=0x7fce44005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:42.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:42.850+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fce4c010070 con 0x7fce54107920 2024-04-25T20:54:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.095+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7fce5402df80 con 0x7fce54107920 2024-04-25T20:54:43.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.307+0000 7fce50ff9700 1 -- 172.21.15.96:0/1179438686 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7fce4c09cc30 con 0x7fce54107920 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fce3c0a3c40 msgr2=0x7fce3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fce3c0a3c40 0x7fce3c0a6100 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fce44005fd0 tx=0x7fce44005db0 comp rx=0 tx=0).stop 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 msgr2=0x7fce5419c900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 0x7fce5419c900 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fce4c007ee0 tx=0x7fce4c00d3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 shutdown_connections 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fce3c0a3c40 0x7fce3c0a6100 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce54107920 0x7fce5419c900 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce54101970 0x7fce54198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:43.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 --2- 172.21.15.96:0/1179438686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fce54102330 0x7fce54198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:43.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 >> 172.21.15.96:0/1179438686 conn(0x7fce540788f0 msgr2=0x7fce540ff9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:43.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 shutdown_connections 2024-04-25T20:54:43.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:43.310+0000 7fce59ef8700 1 -- 172.21.15.96:0/1179438686 wait complete. 2024-04-25T20:54:43.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:43 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1179438686' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T20:54:43.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:43 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1179438686' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T20:54:43.670 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:43 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1179438686' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-04-25T20:54:43.989 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-25T20:54:44.232 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T20:54:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T20:54:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1179438686' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T20:54:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T20:54:44.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:44.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T20:54:44.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:44 smithi097 ceph-mon[32667]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T20:54:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T20:54:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T20:54:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1179438686' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T20:54:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T20:54:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T20:54:44.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:44 smithi139 ceph-mon[33027]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T20:54:44.623 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: pgmap v308: 97 pgs: 97 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-25T20:54:44.624 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-25T20:54:44.624 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1179438686' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-04-25T20:54:44.624 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-25T20:54:44.624 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-25T20:54:44.624 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-25T20:54:44.624 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:44 smithi096 ceph-mon[29288]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-25T20:54:45.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] up:active 2024-04-25T20:54:45.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T20:54:45.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T20:54:45.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: Cluster is now healthy 2024-04-25T20:54:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T20:54:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T20:54:45.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:45 smithi097 ceph-mon[32667]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T20:54:45.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] up:active 2024-04-25T20:54:45.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T20:54:45.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T20:54:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: Cluster is now healthy 2024-04-25T20:54:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T20:54:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T20:54:45.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:45 smithi139 ceph-mon[33027]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T20:54:45.629 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] up:active 2024-04-25T20:54:45.629 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-25T20:54:45.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-25T20:54:45.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: Cluster is now healthy 2024-04-25T20:54:45.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-25T20:54:45.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-25T20:54:45.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:45 smithi096 ceph-mon[29288]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 -- 172.21.15.96:0/2004511064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f906006bb10 msgr2=0x7f906010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- 172.21.15.96:0/2004511064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f906006bb10 0x7f906010f900 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f905800b0a0 tx=0x7f905800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 -- 172.21.15.96:0/2004511064 shutdown_connections 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- 172.21.15.96:0/2004511064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f906006bb10 0x7f906010f900 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- 172.21.15.96:0/2004511064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9060108a30 0x7f9060108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- 172.21.15.96:0/2004511064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006b170 0x7f906006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.006 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 -- 172.21.15.96:0/2004511064 >> 172.21.15.96:0/2004511064 conn(0x7f90600788f0 msgr2=0x7f9060078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 -- 172.21.15.96:0/2004511064 shutdown_connections 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 -- 172.21.15.96:0/2004511064 wait complete. 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 Processor -- start 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 -- start start 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906006b170 0x7f9060199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006bb10 0x7f9060199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.003+0000 7f90682b4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 0x7f906019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90682b4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9060112c30 con 0x7f9060108a30 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90682b4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9060112ab0 con 0x7f906006bb10 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90682b4700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9060112db0 con 0x7f906006b170 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 0x7f906019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 0x7f906019d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35408/0 (socket says 172.21.15.96:35408) 2024-04-25T20:54:46.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 -- 172.21.15.96:0/2197195839 learned_addr learned my addr 172.21.15.96:0/2197195839 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f906584f700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006bb10 0x7f9060199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066050700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906006b170 0x7f9060199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 -- 172.21.15.96:0/2197195839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906006b170 msgr2=0x7f9060199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906006b170 0x7f9060199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 -- 172.21.15.96:0/2197195839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006bb10 msgr2=0x7f9060199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006bb10 0x7f9060199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 -- 172.21.15.96:0/2197195839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f906019e130 con 0x7f9060108a30 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f906584f700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006bb10 0x7f9060199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066050700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906006b170 0x7f9060199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f9066851700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 0x7f906019d9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90580062a0 tx=0x7f9058004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058021070 con 0x7f9060108a30 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f906019e3c0 con 0x7f9060108a30 2024-04-25T20:54:46.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9058007e30 con 0x7f9060108a30 2024-04-25T20:54:46.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058004780 con 0x7f9060108a30 2024-04-25T20:54:46.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.004+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90601a5ca0 con 0x7f9060108a30 2024-04-25T20:54:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.005+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9060069730 con 0x7f9060108a30 2024-04-25T20:54:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.006+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9058007980 con 0x7f9060108a30 2024-04-25T20:54:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.007+0000 7f90577fe700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f904c0a3c40 0x7f904c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.007+0000 7f9066050700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f904c0a3c40 0x7f904c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.007+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f905800f020 con 0x7f9060108a30 2024-04-25T20:54:46.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.007+0000 7f9066050700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f904c0a3c40 0x7f904c0a6100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90500099d0 tx=0x7f9050008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:46.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.012+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f905802a020 con 0x7f9060108a30 2024-04-25T20:54:46.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.253+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f906019a480 con 0x7f9060108a30 2024-04-25T20:54:46.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.342+0000 7f90577fe700 1 -- 172.21.15.96:0/2197195839 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f905809e310 con 0x7f9060108a30 2024-04-25T20:54:46.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f904c0a3c40 msgr2=0x7f904c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:46.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f904c0a3c40 0x7f904c0a6100 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f90500099d0 tx=0x7f9050008040 comp rx=0 tx=0).stop 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 msgr2=0x7f906019d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 0x7f906019d9f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f90580062a0 tx=0x7f9058004c80 comp rx=0 tx=0).stop 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 shutdown_connections 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f904c0a3c40 0x7f904c0a6100 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9060108a30 0x7f906019d9f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f906006b170 0x7f9060199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 --2- 172.21.15.96:0/2197195839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f906006bb10 0x7f9060199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 >> 172.21.15.96:0/2197195839 conn(0x7f90600788f0 msgr2=0x7f90600ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.345+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 shutdown_connections 2024-04-25T20:54:46.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:46.346+0000 7f90682b4700 1 -- 172.21.15.96:0/2197195839 wait complete. 2024-04-25T20:54:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:46 smithi097 ceph-mon[32667]: pgmap v309: 97 pgs: 97 active+clean; 580 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-25T20:54:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:46 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] up:active 2024-04-25T20:54:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:46 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T20:54:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:46 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2197195839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:54:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:46 smithi139 ceph-mon[33027]: pgmap v309: 97 pgs: 97 active+clean; 580 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-25T20:54:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:46 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] up:active 2024-04-25T20:54:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:46 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T20:54:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:46 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2197195839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:54:46.702 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:46 smithi096 ceph-mon[29288]: pgmap v309: 97 pgs: 97 active+clean; 580 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 2 op/s 2024-04-25T20:54:46.702 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:46 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] up:active 2024-04-25T20:54:46.702 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:46 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T20:54:46.702 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:46 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2197195839' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-25T20:54:47.116 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-25T20:54:47.357 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:47.605 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:47 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2197195839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T20:54:47.605 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:47 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T20:54:47.605 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:47 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:54:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:47 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2197195839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T20:54:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:47 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T20:54:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:47 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:54:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:47 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2197195839' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-25T20:54:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:47 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-25T20:54:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:47 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:54:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:48 smithi096 ceph-mon[29288]: pgmap v310: 97 pgs: 97 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.6 KiB/s wr, 5 op/s 2024-04-25T20:54:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:48 smithi097 ceph-mon[32667]: pgmap v310: 97 pgs: 97 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.6 KiB/s wr, 5 op/s 2024-04-25T20:54:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:48 smithi139 ceph-mon[33027]: pgmap v310: 97 pgs: 97 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.6 KiB/s wr, 5 op/s 2024-04-25T20:54:49.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 -- 172.21.15.96:0/849345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980107990 msgr2=0x7f3980107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:49.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 --2- 172.21.15.96:0/849345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980107990 0x7f3980107d70 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f397c009990 tx=0x7f397c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:49.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 -- 172.21.15.96:0/849345984 shutdown_connections 2024-04-25T20:54:49.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 --2- 172.21.15.96:0/849345984 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980102300 0x7f398010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:49.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 --2- 172.21.15.96:0/849345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980107990 0x7f3980107d70 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:49.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 --2- 172.21.15.96:0/849345984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3980101940 0x7f3980101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:49.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 -- 172.21.15.96:0/849345984 >> 172.21.15.96:0/849345984 conn(0x7f39800788f0 msgr2=0x7f3980078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:49.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 -- 172.21.15.96:0/849345984 shutdown_connections 2024-04-25T20:54:49.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.098+0000 7f39883d8700 1 -- 172.21.15.96:0/849345984 wait complete. 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 Processor -- start 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 -- start start 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980101940 0x7f398019a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 0x7f398019a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3980107990 0x7f39801a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3980111b70 con 0x7f3980102300 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f39801119f0 con 0x7f3980107990 2024-04-25T20:54:49.102 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f39883d8700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3980111cf0 con 0x7f3980101940 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f3985973700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 0x7f398019a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f3985973700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 0x7f398019a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54720/0 (socket says 172.21.15.96:54720) 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.099+0000 7f3985973700 1 -- 172.21.15.96:0/3086505250 learned_addr learned my addr 172.21.15.96:0/3086505250 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3985973700 1 -- 172.21.15.96:0/3086505250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980101940 msgr2=0x7f398019a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3986174700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980101940 0x7f398019a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3985973700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980101940 0x7f398019a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3985973700 1 -- 172.21.15.96:0/3086505250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3980107990 msgr2=0x7f39801a50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3986975700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3980107990 0x7f39801a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3985973700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3980107990 0x7f39801a50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3985973700 1 -- 172.21.15.96:0/3086505250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39801a5610 con 0x7f3980102300 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3986174700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980101940 0x7f398019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f3985973700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 0x7f398019a590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f397000b530 tx=0x7f397000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970015020 con 0x7f3980102300 2024-04-25T20:54:49.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3970011ce0 con 0x7f3980102300 2024-04-25T20:54:49.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39801a5840 con 0x7f3980102300 2024-04-25T20:54:49.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970010d20 con 0x7f3980102300 2024-04-25T20:54:49.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.100+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39801a5d80 con 0x7f3980102300 2024-04-25T20:54:49.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.101+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3980069730 con 0x7f3980102300 2024-04-25T20:54:49.104 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.102+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3970018020 con 0x7f3980102300 2024-04-25T20:54:49.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.103+0000 7f39777fe700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f396c0a3c40 0x7f396c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:49.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.103+0000 7f3986174700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f396c0a3c40 0x7f396c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:49.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.103+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f39700cfaf0 con 0x7f3980102300 2024-04-25T20:54:49.105 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.103+0000 7f3986174700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f396c0a3c40 0x7f396c0a6100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b5c0 tx=0x7f397c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:49.108 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.108+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f397001f020 con 0x7f3980102300 2024-04-25T20:54:49.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:49.354+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f398002df80 con 0x7f3980102300 2024-04-25T20:54:50.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.359+0000 7f39777fe700 1 -- 172.21.15.96:0/3086505250 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f397009c620 con 0x7f3980102300 2024-04-25T20:54:50.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f396c0a3c40 msgr2=0x7f396c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:50.363 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f396c0a3c40 0x7f396c0a6100 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f397c00b5c0 tx=0x7f397c005fb0 comp rx=0 tx=0).stop 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 msgr2=0x7f398019a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 0x7f398019a590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f397000b530 tx=0x7f397000b8f0 comp rx=0 tx=0).stop 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 shutdown_connections 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f396c0a3c40 0x7f396c0a6100 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3980107990 0x7f39801a50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3980101940 0x7f398019a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 --2- 172.21.15.96:0/3086505250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3980102300 0x7f398019a590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 >> 172.21.15.96:0/3086505250 conn(0x7f39800788f0 msgr2=0x7f39801007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 shutdown_connections 2024-04-25T20:54:50.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:50.362+0000 7f39883d8700 1 -- 172.21.15.96:0/3086505250 wait complete. 2024-04-25T20:54:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:50 smithi097 ceph-mon[32667]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T20:54:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:50 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3086505250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T20:54:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:50 smithi139 ceph-mon[33027]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T20:54:50.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:50 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3086505250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T20:54:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:50 smithi096 ceph-mon[29288]: pgmap v311: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-04-25T20:54:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:50 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3086505250' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-25T20:54:51.245 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-04-25T20:54:51.245 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-25T20:54:51.492 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:51 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3086505250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T20:54:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:51 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:54:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:51 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3086505250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T20:54:51.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:51 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:54:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:51 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3086505250' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-25T20:54:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:51 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:54:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:52 smithi139 ceph-mon[33027]: pgmap v312: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:54:52.643 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:52 smithi096 ceph-mon[29288]: pgmap v312: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:54:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:52 smithi097 ceph-mon[32667]: pgmap v312: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:54:53.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 -- 172.21.15.96:0/148694338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718101970 msgr2=0x7f7718101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:53.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 --2- 172.21.15.96:0/148694338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718101970 0x7f7718101df0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7708009990 tx=0x7f7708009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 -- 172.21.15.96:0/148694338 shutdown_connections 2024-04-25T20:54:53.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 --2- 172.21.15.96:0/148694338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7718102330 0x7f771810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 --2- 172.21.15.96:0/148694338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718107920 0x7f7718107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 --2- 172.21.15.96:0/148694338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718101970 0x7f7718101df0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 -- 172.21.15.96:0/148694338 >> 172.21.15.96:0/148694338 conn(0x7f77180779b0 msgr2=0x7f7718077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:53.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 -- 172.21.15.96:0/148694338 shutdown_connections 2024-04-25T20:54:53.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.173+0000 7f771fea5700 1 -- 172.21.15.96:0/148694338 wait complete. 2024-04-25T20:54:53.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 Processor -- start 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 -- start start 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7718101970 0x7f77181034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718102330 0x7f7718103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 0x7f7718103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7718111bc0 con 0x7f7718107920 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7718111a40 con 0x7f7718101970 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771fea5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7718111d40 con 0x7f7718102330 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 0x7f7718103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771d440700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718102330 0x7f7718103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 0x7f7718103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54736/0 (socket says 172.21.15.96:54736) 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771d440700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718102330 0x7f7718103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:48258/0 (socket says 172.21.15.96:48258) 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 -- 172.21.15.96:0/800193804 learned_addr learned my addr 172.21.15.96:0/800193804 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771dc41700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7718101970 0x7f77181034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:53.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 -- 172.21.15.96:0/800193804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718102330 msgr2=0x7f7718103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718102330 0x7f7718103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 -- 172.21.15.96:0/800193804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7718101970 msgr2=0x7f77181034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7718101970 0x7f77181034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.174+0000 7f771e442700 1 -- 172.21.15.96:0/800193804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77181a7770 con 0x7f7718107920 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.175+0000 7f771e442700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 0x7f7718103f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f771000b970 tx=0x7f771000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.175+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771000d860 con 0x7f7718107920 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.175+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77181a7a60 con 0x7f7718107920 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.175+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f771000e430 con 0x7f7718107920 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.175+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7710017660 con 0x7f7718107920 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.175+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77181a7f70 con 0x7f7718107920 2024-04-25T20:54:53.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.176+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7718103060 con 0x7f7718107920 2024-04-25T20:54:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.177+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f771000da00 con 0x7f7718107920 2024-04-25T20:54:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.177+0000 7f770effd700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f77040a3c40 0x7f77040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.177+0000 7f771dc41700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f77040a3c40 0x7f77040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.178+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f77100d0070 con 0x7f7718107920 2024-04-25T20:54:53.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.178+0000 7f771dc41700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f77040a3c40 0x7f77040a6100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7714009a10 tx=0x7f7714009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:53.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.181+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7710013030 con 0x7f7718107920 2024-04-25T20:54:53.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.412+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7718062e80 con 0x7f77040a3c40 2024-04-25T20:54:53.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.435+0000 7f770effd700 1 -- 172.21.15.96:0/800193804 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7718062e80 con 0x7f77040a3c40 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f77040a3c40 msgr2=0x7f77040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f77040a3c40 0x7f77040a6100 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7714009a10 tx=0x7f7714009190 comp rx=0 tx=0).stop 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 msgr2=0x7f7718103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 0x7f7718103f70 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f771000b970 tx=0x7f771000bc80 comp rx=0 tx=0).stop 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 shutdown_connections 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f77040a3c40 0x7f77040a6100 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7718107920 0x7f7718103f70 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7718101970 0x7f77181034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 --2- 172.21.15.96:0/800193804 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7718102330 0x7f7718103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 >> 172.21.15.96:0/800193804 conn(0x7f77180779b0 msgr2=0x7f771806b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:53.439 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 shutdown_connections 2024-04-25T20:54:53.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:53.438+0000 7f771fea5700 1 -- 172.21.15.96:0/800193804 wait complete. 2024-04-25T20:54:53.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:54:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:54:53.416+0000 7f59ecf35700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:54:53.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:54:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:54:53.416+0000 7f59ecf35700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:54:53.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:54:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:54:53.416+0000 7f59ecf35700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:54:53.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:54:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:54:53.416+0000 7f59ecf35700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:54:53.750 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:54:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:54:53.416+0000 7f59ecf35700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:54:54.153 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-25T20:54:54.392 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:54.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:54 smithi096 ceph-mon[29288]: pgmap v313: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T20:54:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:54 smithi139 ceph-mon[33027]: pgmap v313: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T20:54:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:54 smithi097 ceph-mon[32667]: pgmap v313: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-25T20:54:55.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:55 smithi096 ceph-mon[29288]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:55 smithi097 ceph-mon[32667]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:55 smithi139 ceph-mon[33027]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:56.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 -- 172.21.15.96:0/3064960217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 msgr2=0x7f4aa0101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 --2- 172.21.15.96:0/3064960217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa0101de0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4a94009990 tx=0x7f4a94009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 -- 172.21.15.96:0/3064960217 shutdown_connections 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 --2- 172.21.15.96:0/3064960217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa0102320 0x7f4aa010e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 --2- 172.21.15.96:0/3064960217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa0107930 0x7f4aa0107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 --2- 172.21.15.96:0/3064960217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa0101de0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 -- 172.21.15.96:0/3064960217 >> 172.21.15.96:0/3064960217 conn(0x7f4aa0078870 msgr2=0x7f4aa0078c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 -- 172.21.15.96:0/3064960217 shutdown_connections 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 -- 172.21.15.96:0/3064960217 wait complete. 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 Processor -- start 2024-04-25T20:54:56.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.148+0000 7f4aa6948700 1 -- start start 2024-04-25T20:54:56.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:56.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa0102320 0x7f4aa01985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:56.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa0107930 0x7f4aa019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:56.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa0111b40 con 0x7f4aa0101980 2024-04-25T20:54:56.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa01119c0 con 0x7f4aa0107930 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4aa0111cc0 con 0x7f4aa0102320 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa01980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54756/0 (socket says 172.21.15.96:54756) 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 -- 172.21.15.96:0/2082168832 learned_addr learned my addr 172.21.15.96:0/2082168832 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9f7fe700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa0102320 0x7f4aa01985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 -- 172.21.15.96:0/2082168832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa0102320 msgr2=0x7f4aa01985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa0102320 0x7f4aa01985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa4ee5700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa0107930 0x7f4aa019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 -- 172.21.15.96:0/2082168832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa0107930 msgr2=0x7f4aa019c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa0107930 0x7f4aa019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 -- 172.21.15.96:0/2082168832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4aa019d0c0 con 0x7f4aa0101980 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9ffff700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa01980b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000b530 tx=0x7f4a9000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90015020 con 0x7f4aa0101980 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4aa019d3b0 con 0x7f4aa0101980 2024-04-25T20:54:56.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a90011ce0 con 0x7f4aa0101980 2024-04-25T20:54:56.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a90010d20 con 0x7f4aa0101980 2024-04-25T20:54:56.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.149+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4aa01a4c30 con 0x7f4aa0101980 2024-04-25T20:54:56.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.150+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4aa0103070 con 0x7f4aa0101980 2024-04-25T20:54:56.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.152+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4a90018020 con 0x7f4aa0101980 2024-04-25T20:54:56.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.152+0000 7f4a9d7fa700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a880a3cd0 0x7f4a880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:56.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.152+0000 7f4a9f7fe700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a880a3cd0 0x7f4a880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:56.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.152+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f4a900d00a0 con 0x7f4aa0101980 2024-04-25T20:54:56.155 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.153+0000 7f4a9f7fe700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a880a3cd0 0x7f4a880a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a94006010 tx=0x7f4a9401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:56.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.156+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a9001f020 con 0x7f4aa0101980 2024-04-25T20:54:56.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.398+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4aa0062e80 con 0x7f4a880a3cd0 2024-04-25T20:54:56.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.424+0000 7f4a9d7fa700 1 -- 172.21.15.96:0/2082168832 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4aa0062e80 con 0x7f4a880a3cd0 2024-04-25T20:54:56.425 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:56 smithi096 ceph-mon[29288]: pgmap v314: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T20:54:56.425 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:56 smithi096 ceph-mon[29288]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T20:54:56.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a880a3cd0 msgr2=0x7f4a880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a880a3cd0 0x7f4a880a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4a94006010 tx=0x7f4a9401a040 comp rx=0 tx=0).stop 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 msgr2=0x7f4aa01980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa01980b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4a9000b530 tx=0x7f4a9000b8f0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 shutdown_connections 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a880a3cd0 0x7f4a880a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4aa0107930 0x7f4aa019c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4aa0101980 0x7f4aa01980b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 --2- 172.21.15.96:0/2082168832 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4aa0102320 0x7f4aa01985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 >> 172.21.15.96:0/2082168832 conn(0x7f4aa0078870 msgr2=0x7f4aa00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 shutdown_connections 2024-04-25T20:54:56.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:56.428+0000 7f4aa6948700 1 -- 172.21.15.96:0/2082168832 wait complete. 2024-04-25T20:54:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:56 smithi097 ceph-mon[32667]: pgmap v314: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T20:54:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:56 smithi097 ceph-mon[32667]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T20:54:56.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:56 smithi139 ceph-mon[33027]: pgmap v314: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T20:54:56.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:56 smithi139 ceph-mon[33027]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-25T20:54:57.159 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-25T20:54:57.389 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:57 smithi096 ceph-mon[29288]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:57.390 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:54:57 smithi096 ceph-mon[29288]: pgmap v315: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-25T20:54:57.413 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:54:57.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:57 smithi097 ceph-mon[32667]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:57.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:54:57 smithi097 ceph-mon[32667]: pgmap v315: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-25T20:54:57.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:57 smithi139 ceph-mon[33027]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:54:57.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:54:57 smithi139 ceph-mon[33027]: pgmap v315: 97 pgs: 97 active+clean; 582 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.3 KiB/s wr, 11 op/s 2024-04-25T20:54:59.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 -- 172.21.15.96:0/2552324686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 msgr2=0x7f6698108eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:59.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 --2- 172.21.15.96:0/2552324686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f6698108eb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6688009990 tx=0x7f6688009ca0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 -- 172.21.15.96:0/2552324686 shutdown_connections 2024-04-25T20:54:59.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 --2- 172.21.15.96:0/2552324686 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66981019d0 0x7f669810f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.434 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 --2- 172.21.15.96:0/2552324686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f6698108eb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 --2- 172.21.15.96:0/2552324686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6698101030 0x7f6698101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 -- 172.21.15.96:0/2552324686 >> 172.21.15.96:0/2552324686 conn(0x7f6698077fb0 msgr2=0x7f66980ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 -- 172.21.15.96:0/2552324686 shutdown_connections 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.395+0000 7f669da30700 1 -- 172.21.15.96:0/2552324686 wait complete. 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 Processor -- start 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 -- start start 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6698101030 0x7f6698199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66981019d0 0x7f66981996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f669819da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112c60 con 0x7f6698108ad0 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112ae0 con 0x7f6698101030 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f669da30700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6698112de0 con 0x7f66981019d0 2024-04-25T20:54:59.435 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f66977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f669819da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f66977fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f669819da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55490/0 (socket says 172.21.15.96:55490) 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f66977fe700 1 -- 172.21.15.96:0/1031567262 learned_addr learned my addr 172.21.15.96:0/1031567262 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f6696ffd700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6698101030 0x7f6698199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f66977fe700 1 -- 172.21.15.96:0/1031567262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66981019d0 msgr2=0x7f66981996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.396+0000 7f66977fe700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66981019d0 0x7f66981996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f66977fe700 1 -- 172.21.15.96:0/1031567262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6698101030 msgr2=0x7f6698199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f66977fe700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6698101030 0x7f6698199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f66977fe700 1 -- 172.21.15.96:0/1031567262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669819e180 con 0x7f6698108ad0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f66977fe700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f669819da40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6690007ee0 tx=0x7f669000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669000dbd0 con 0x7f6698108ad0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669819e470 con 0x7f6698108ad0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f669000dd70 con 0x7f6698108ad0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6690005990 con 0x7f6698108ad0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.397+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66981a5cf0 con 0x7f6698108ad0 2024-04-25T20:54:59.436 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.398+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6698104350 con 0x7f6698108ad0 2024-04-25T20:54:59.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.399+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f669001a020 con 0x7f6698108ad0 2024-04-25T20:54:59.437 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.400+0000 7f669ca2e700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f66840a3cd0 0x7f66840a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:54:59.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.400+0000 7f6696ffd700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f66840a3cd0 0x7f66840a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:54:59.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.400+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f66900d02d0 con 0x7f6698108ad0 2024-04-25T20:54:59.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.400+0000 7f6696ffd700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f66840a3cd0 0x7f66840a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f668800b5c0 tx=0x7f6688005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:54:59.438 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.403+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6690010070 con 0x7f6698108ad0 2024-04-25T20:54:59.635 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.635+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6698062e80 con 0x7f66840a3cd0 2024-04-25T20:54:59.641 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.641+0000 7f669ca2e700 1 -- 172.21.15.96:0/1031567262 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6698062e80 con 0x7f66840a3cd0 2024-04-25T20:54:59.641 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_0/325518f6-b5f2-45c3-a87e-edc9bcb63c06 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.643+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f66840a3cd0 msgr2=0x7f66840a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.643+0000 7f669da30700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f66840a3cd0 0x7f66840a6190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f668800b5c0 tx=0x7f6688005960 comp rx=0 tx=0).stop 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.643+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 msgr2=0x7f669819da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.643+0000 7f669da30700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f669819da40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f6690007ee0 tx=0x7f669000d3b0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 shutdown_connections 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f66840a3cd0 0x7f66840a6190 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6698108ad0 0x7f669819da40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6698101030 0x7f6698199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 --2- 172.21.15.96:0/1031567262 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f66981019d0 0x7f66981996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 >> 172.21.15.96:0/1031567262 conn(0x7f6698077fb0 msgr2=0x7f66980ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 shutdown_connections 2024-04-25T20:54:59.644 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:54:59.644+0000 7f669da30700 1 -- 172.21.15.96:0/1031567262 wait complete. 2024-04-25T20:55:00.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-25T20:55:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:00 smithi097 ceph-mon[32667]: pgmap v316: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-25T20:55:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:00 smithi139 ceph-mon[33027]: pgmap v316: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-25T20:55:00.728 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:00.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:00 smithi096 ceph-mon[29288]: pgmap v316: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.3 KiB/s wr, 10 op/s 2024-04-25T20:55:01.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:01 smithi096 ceph-mon[29288]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:01.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:01 smithi097 ceph-mon[32667]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:01.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:01 smithi139 ceph-mon[33027]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:02.411 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 -- 172.21.15.96:0/4105249853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 msgr2=0x7f7904105d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 --2- 172.21.15.96:0/4105249853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f7904105d20 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f78f40099c0 tx=0x7f78f4009cd0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 -- 172.21.15.96:0/4105249853 shutdown_connections 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 --2- 172.21.15.96:0/4105249853 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7904106400 0x7f790410dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 --2- 172.21.15.96:0/4105249853 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7904101320 0x7f7904101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 --2- 172.21.15.96:0/4105249853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f7904105d20 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 -- 172.21.15.96:0/4105249853 >> 172.21.15.96:0/4105249853 conn(0x7f7904077f40 msgr2=0x7f79040ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 -- 172.21.15.96:0/4105249853 shutdown_connections 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.410+0000 7f790c1db700 1 -- 172.21.15.96:0/4105249853 wait complete. 2024-04-25T20:55:02.412 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 Processor -- start 2024-04-25T20:55:02.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 -- start start 2024-04-25T20:55:02.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7904101320 0x7f79040fd380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:02.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f79040fd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f7909776700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f79040fd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f7909776700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f79040fd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55512/0 (socket says 172.21.15.96:55512) 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f7909776700 1 -- 172.21.15.96:0/1717802091 learned_addr learned my addr 172.21.15.96:0/1717802091 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7904106400 0x7f79040fde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7904111ba0 con 0x7f7904101cd0 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7904111a20 con 0x7f7904106400 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7904111d20 con 0x7f7904101320 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f7909776700 1 -- 172.21.15.96:0/1717802091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7904101320 msgr2=0x7f79040fd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f7909776700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7904101320 0x7f79040fd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.411+0000 7f7909776700 1 -- 172.21.15.96:0/1717802091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7904106400 msgr2=0x7f79040fde00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f7909776700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7904106400 0x7f79040fde00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f7909776700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79041abba0 con 0x7f7904101cd0 2024-04-25T20:55:02.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f7909776700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f79040fd8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009f50 tx=0x7f78f40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:02.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f4027070 con 0x7f7904101cd0 2024-04-25T20:55:02.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78f4004140 con 0x7f7904101cd0 2024-04-25T20:55:02.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79041abdd0 con 0x7f7904101cd0 2024-04-25T20:55:02.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78f400f460 con 0x7f7904101cd0 2024-04-25T20:55:02.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.412+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79041ac1c0 con 0x7f7904101cd0 2024-04-25T20:55:02.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.413+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7904102800 con 0x7f7904101cd0 2024-04-25T20:55:02.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.414+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f78f402a070 con 0x7f7904101cd0 2024-04-25T20:55:02.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.414+0000 7f78faffd700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f78f00a3cd0 0x7f78f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:02.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.415+0000 7f7909f77700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f78f00a3cd0 0x7f78f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:02.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.415+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f78f40d1850 con 0x7f7904101cd0 2024-04-25T20:55:02.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.415+0000 7f7909f77700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f78f00a3cd0 0x7f78f00a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f79000097b0 tx=0x7f7900006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:02.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.418+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78f4031070 con 0x7f7904101cd0 2024-04-25T20:55:02.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:02 smithi096 ceph-mon[29288]: pgmap v317: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T20:55:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:02 smithi097 ceph-mon[32667]: pgmap v317: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T20:55:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:02 smithi139 ceph-mon[33027]: pgmap v317: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-04-25T20:55:02.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.653+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f7904105340 con 0x7f78f00a3cd0 2024-04-25T20:55:02.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.679+0000 7f78faffd700 1 -- 172.21.15.96:0/1717802091 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7904105340 con 0x7f78f00a3cd0 2024-04-25T20:55:02.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.681+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f78f00a3cd0 msgr2=0x7f78f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.681+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f78f00a3cd0 0x7f78f00a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f79000097b0 tx=0x7f7900006d20 comp rx=0 tx=0).stop 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.681+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 msgr2=0x7f79040fd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.681+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f79040fd8c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f78f4009f50 tx=0x7f78f40037c0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 shutdown_connections 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f78f00a3cd0 0x7f78f00a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7904106400 0x7f79040fde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7904101320 0x7f79040fd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 --2- 172.21.15.96:0/1717802091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7904101cd0 0x7f79040fd8c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 >> 172.21.15.96:0/1717802091 conn(0x7f7904077f40 msgr2=0x7f79041107a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 shutdown_connections 2024-04-25T20:55:02.683 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:02.682+0000 7f790c1db700 1 -- 172.21.15.96:0/1717802091 wait complete. 2024-04-25T20:55:03.360 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-25T20:55:03.599 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:04 smithi096 ceph-mon[29288]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:04 smithi096 ceph-mon[29288]: pgmap v318: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-25T20:55:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:04 smithi097 ceph-mon[32667]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:04 smithi097 ceph-mon[32667]: pgmap v318: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-25T20:55:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:04 smithi139 ceph-mon[33027]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:04 smithi139 ceph-mon[33027]: pgmap v318: 97 pgs: 97 active+clean; 602 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.3 KiB/s wr, 7 op/s 2024-04-25T20:55:05.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3120258985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784101970 msgr2=0x7fa784101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:05.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3120258985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784101970 0x7fa784101df0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa774009990 tx=0x7fa774009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3120258985 shutdown_connections 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3120258985 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784102330 0x7fa78410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3120258985 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa784107920 0x7fa784107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3120258985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784101970 0x7fa784101df0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3120258985 >> 172.21.15.96:0/3120258985 conn(0x7fa784078910 msgr2=0x7fa784078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3120258985 shutdown_connections 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.264+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3120258985 wait complete. 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 Processor -- start 2024-04-25T20:55:05.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 -- start start 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa784101970 0x7fa784199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 0x7fa7841995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784107920 0x7fa78419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa784112c00 con 0x7fa784102330 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa784112a80 con 0x7fa784107920 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa78b5f7700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa784112d80 con 0x7fa784101970 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 0x7fa7841995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 0x7fa7841995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55534/0 (socket says 172.21.15.96:55534) 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 -- 172.21.15.96:0/3490967645 learned_addr learned my addr 172.21.15.96:0/3490967645 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa789b94700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784107920 0x7fa78419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 -- 172.21.15.96:0/3490967645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa784101970 msgr2=0x7fa784199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa784101970 0x7fa784199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 -- 172.21.15.96:0/3490967645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784107920 msgr2=0x7fa78419d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784107920 0x7fa78419d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 -- 172.21.15.96:0/3490967645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78419e080 con 0x7fa784102330 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa789b94700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784107920 0x7fa78419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.265+0000 7fa788b92700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 0x7fa7841995b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7740057c0 tx=0x7fa7740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.266+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774027070 con 0x7fa784102330 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.266+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78419e310 con 0x7fa784102330 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.266+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa774004020 con 0x7fa784102330 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.266+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa774022bf0 con 0x7fa784102330 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.266+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7841a5bf0 con 0x7fa784102330 2024-04-25T20:55:05.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.267+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa784069730 con 0x7fa784102330 2024-04-25T20:55:05.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.268+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa77402a070 con 0x7fa784102330 2024-04-25T20:55:05.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.268+0000 7fa77a7fc700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa7700a3cd0 0x7fa7700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:05.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.268+0000 7fa789393700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa7700a3cd0 0x7fa7700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:05.270 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.268+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7fa7740d1300 con 0x7fa784102330 2024-04-25T20:55:05.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.269+0000 7fa789393700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa7700a3cd0 0x7fa7700a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa780007fb0 tx=0x7fa780007570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:05.274 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.273+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa77405eff0 con 0x7fa784102330 2024-04-25T20:55:05.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.510+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa784062e80 con 0x7fa7700a3cd0 2024-04-25T20:55:05.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.514+0000 7fa77a7fc700 1 -- 172.21.15.96:0/3490967645 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa784062e80 con 0x7fa7700a3cd0 2024-04-25T20:55:05.515 INFO:teuthology.orchestra.run.smithi096.stdout:/volumes/qa/sv_1/9e3c18f5-af88-4dc8-93e6-d6c5693be9b9 2024-04-25T20:55:05.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa7700a3cd0 msgr2=0x7fa7700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:05.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa7700a3cd0 0x7fa7700a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa780007fb0 tx=0x7fa780007570 comp rx=0 tx=0).stop 2024-04-25T20:55:05.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 msgr2=0x7fa7841995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:05.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 0x7fa7841995b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7740057c0 tx=0x7fa7740057f0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 shutdown_connections 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa7700a3cd0 0x7fa7700a6190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa784107920 0x7fa78419d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa784101970 0x7fa784199070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 --2- 172.21.15.96:0/3490967645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa784102330 0x7fa7841995b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 >> 172.21.15.96:0/3490967645 conn(0x7fa784078910 msgr2=0x7fa7840ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 shutdown_connections 2024-04-25T20:55:05.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:05.516+0000 7fa78b5f7700 1 -- 172.21.15.96:0/3490967645 wait complete. 2024-04-25T20:55:06.209 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:55:06.450 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:06.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:06 smithi096 ceph-mon[29288]: pgmap v319: 97 pgs: 97 active+clean; 622 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-04-25T20:55:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:06 smithi097 ceph-mon[32667]: pgmap v319: 97 pgs: 97 active+clean; 622 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-04-25T20:55:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:06 smithi139 ceph-mon[33027]: pgmap v319: 97 pgs: 97 active+clean; 622 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.1 KiB/s wr, 8 op/s 2024-04-25T20:55:07.596 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:07 smithi096 ceph-mon[29288]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:07.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:07 smithi097 ceph-mon[32667]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:07.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:07 smithi139 ceph-mon[33027]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:55:08.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 -- 172.21.15.96:0/4038183635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c108a50 msgr2=0x7f2a8c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:08.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 --2- 172.21.15.96:0/4038183635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c108a50 0x7f2a8c108e30 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009990 tx=0x7f2a7c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 -- 172.21.15.96:0/4038183635 shutdown_connections 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 --2- 172.21.15.96:0/4038183635 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c06bb70 0x7f2a8c10f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 --2- 172.21.15.96:0/4038183635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c108a50 0x7f2a8c108e30 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 --2- 172.21.15.96:0/4038183635 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c06b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 -- 172.21.15.96:0/4038183635 >> 172.21.15.96:0/4038183635 conn(0x7f2a8c077990 msgr2=0x7f2a8c077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 -- 172.21.15.96:0/4038183635 shutdown_connections 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.072+0000 7f2a93bce700 1 -- 172.21.15.96:0/4038183635 wait complete. 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 Processor -- start 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 -- start start 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:08.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06bb70 0x7f2a8c1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c108a50 0x7f2a8c19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112c90 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112b10 con 0x7f2a8c108a50 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a93bce700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8c112e10 con 0x7f2a8c06bb70 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55370/0 (socket says 172.21.15.96:55370) 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 -- 172.21.15.96:0/3258162847 learned_addr learned my addr 172.21.15.96:0/3258162847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9216b700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c108a50 0x7f2a8c19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 -- 172.21.15.96:0/3258162847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06bb70 msgr2=0x7f2a8c1996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a91169700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06bb70 0x7f2a8c1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06bb70 0x7f2a8c1996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 -- 172.21.15.96:0/3258162847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c108a50 msgr2=0x7f2a8c19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:08.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.073+0000 7f2a9196a700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c108a50 0x7f2a8c19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a9196a700 1 -- 172.21.15.96:0/3258162847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a8c19e170 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a9216b700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c108a50 0x7f2a8c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a91169700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06bb70 0x7f2a8c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a9196a700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c199160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c000c00 tx=0x7f2a7c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c027070 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a8c19e400 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a7c0041d0 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c00f460 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.074+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a8c1a5ce0 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.075+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a8c10afe0 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.076+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2a7c02a070 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.076+0000 7f2a82ffd700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2a780a3cd0 0x7f2a780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:08.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.076+0000 7f2a91169700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2a780a3cd0 0x7f2a780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:08.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.076+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 7979+0+0 (secure 0 0 0) 0x7f2a7c0d17c0 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.077+0000 7f2a91169700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2a780a3cd0 0x7f2a780a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c078020 tx=0x7f2a88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:08.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.080+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2a7c031070 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.323+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a8c19a4c0 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.325 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.324+0000 7f2a82ffd700 1 -- 172.21.15.96:0/3258162847 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f2a7c013070 con 0x7f2a8c06b1b0 2024-04-25T20:55:08.325 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:08.325 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:54:49.357397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:55:08.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2a780a3cd0 msgr2=0x7f2a780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:08.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2a780a3cd0 0x7f2a780a6190 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c078020 tx=0x7f2a88008040 comp rx=0 tx=0).stop 2024-04-25T20:55:08.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 msgr2=0x7f2a8c199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:08.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c199160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c000c00 tx=0x7f2a7c0047b0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 shutdown_connections 2024-04-25T20:55:08.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2a780a3cd0 0x7f2a780a6190 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a8c108a50 0x7f2a8c19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2a8c06b1b0 0x7f2a8c199160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 --2- 172.21.15.96:0/3258162847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2a8c06bb70 0x7f2a8c1996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 >> 172.21.15.96:0/3258162847 conn(0x7f2a8c077990 msgr2=0x7f2a8c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 shutdown_connections 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:08.326+0000 7f2a93bce700 1 -- 172.21.15.96:0/3258162847 wait complete. 2024-04-25T20:55:08.328 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:55:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:08 smithi097 ceph-mon[32667]: pgmap v320: 97 pgs: 97 active+clean; 622 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T20:55:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:08 smithi139 ceph-mon[33027]: pgmap v320: 97 pgs: 97 active+clean; 622 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T20:55:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:08 smithi096 ceph-mon[29288]: pgmap v320: 97 pgs: 97 active+clean; 622 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.2 KiB/s wr, 7 op/s 2024-04-25T20:55:09.005 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:55:09.245 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:09.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3258162847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:55:09.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:09 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3258162847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:55:09.614 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T20:55:09.615 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T20:55:09.615 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T20:55:09.615 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T20:55:09.615 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T20:55:09.615 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:09 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3258162847' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-04-25T20:55:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:09 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]: dispatch 2024-04-25T20:55:10.410 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: pgmap v321: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-04-25T20:55:10.410 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T20:55:10.410 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T20:55:10.410 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T20:55:10.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T20:55:10.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T20:55:10.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-25T20:55:10.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:10 smithi096 ceph-mon[29288]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T20:55:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: pgmap v321: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-04-25T20:55:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T20:55:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T20:55:10.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T20:55:10.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T20:55:10.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T20:55:10.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-25T20:55:10.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:10 smithi097 ceph-mon[32667]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T20:55:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: pgmap v321: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.1 KiB/s wr, 9 op/s 2024-04-25T20:55:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-04-25T20:55:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2024-04-25T20:55:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-25T20:55:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-04-25T20:55:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-04-25T20:55:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.33", "id": [5, 9]}]': finished 2024-04-25T20:55:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:10 smithi139 ceph-mon[33027]: osdmap e92: 12 total, 12 up, 12 in 2024-04-25T20:55:10.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 -- 172.21.15.96:0/1973493391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 msgr2=0x7fd26c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:10.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 --2- 172.21.15.96:0/1973493391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c102f40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009990 tx=0x7fd25c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:10.931 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 -- 172.21.15.96:0/1973493391 shutdown_connections 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 --2- 172.21.15.96:0/1973493391 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd26c103480 0x7fd26c10f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 --2- 172.21.15.96:0/1973493391 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26c108a70 0x7fd26c108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 --2- 172.21.15.96:0/1973493391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c102f40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 -- 172.21.15.96:0/1973493391 >> 172.21.15.96:0/1973493391 conn(0x7fd26c079bb0 msgr2=0x7fd26c079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 -- 172.21.15.96:0/1973493391 shutdown_connections 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 -- 172.21.15.96:0/1973493391 wait complete. 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 Processor -- start 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.928+0000 7fd2730a8700 1 -- start start 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd26c103480 0x7fd26c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:10.932 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26c108a70 0x7fd26c19da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c112ca0 con 0x7fd26c102ac0 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c112b20 con 0x7fd26c108a70 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd26c112e20 con 0x7fd26c103480 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c199170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55378/0 (socket says 172.21.15.96:55378) 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 -- 172.21.15.96:0/477265052 learned_addr learned my addr 172.21.15.96:0/477265052 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd263fff700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd26c103480 0x7fd26c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 -- 172.21.15.96:0/477265052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd26c103480 msgr2=0x7fd26c1996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd271645700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26c108a70 0x7fd26c19da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd26c103480 0x7fd26c1996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 -- 172.21.15.96:0/477265052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26c108a70 msgr2=0x7fd26c19da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:10.933 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26c108a70 0x7fd26c19da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 -- 172.21.15.96:0/477265052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26c19e180 con 0x7fd26c102ac0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd270e44700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c199170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c920 tx=0x7fd26800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268007b00 con 0x7fd26c102ac0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd268004ce0 con 0x7fd26c102ac0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd268005550 con 0x7fd26c102ac0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.929+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26c19e470 con 0x7fd26c102ac0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.930+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26c1a5cf0 con 0x7fd26c102ac0 2024-04-25T20:55:10.934 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.930+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd26c069730 con 0x7fd26c102ac0 2024-04-25T20:55:10.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.931+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd268007ca0 con 0x7fd26c102ac0 2024-04-25T20:55:10.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.932+0000 7fd261ffb700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd2540a3cd0 0x7fd2540a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:10.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.932+0000 7fd263fff700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd2540a3cd0 0x7fd2540a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:10.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.932+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fd2680d0390 con 0x7fd26c102ac0 2024-04-25T20:55:10.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.932+0000 7fd263fff700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd2540a3cd0 0x7fd2540a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0098e0 tx=0x7fd25c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:10.937 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:10.937+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd268014030 con 0x7fd26c102ac0 2024-04-25T20:55:11.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.136+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd26c02df80 con 0x7fd26c102ac0 2024-04-25T20:55:11.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.138+0000 7fd261ffb700 1 -- 172.21.15.96:0/477265052 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22523 (secure 0 0 0) 0x7fd26809cd10 con 0x7fd26c102ac0 2024-04-25T20:55:11.139 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:11.139 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":93,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:55:10.320396+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:55:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd2540a3cd0 msgr2=0x7fd2540a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd2540a3cd0 0x7fd2540a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd25c0098e0 tx=0x7fd25c005960 comp rx=0 tx=0).stop 2024-04-25T20:55:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 msgr2=0x7fd26c199170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c199170 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd26800c920 tx=0x7fd26800cc30 comp rx=0 tx=0).stop 2024-04-25T20:55:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 shutdown_connections 2024-04-25T20:55:11.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd2540a3cd0 0x7fd2540a6190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd26c108a70 0x7fd26c19da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd26c102ac0 0x7fd26c199170 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 --2- 172.21.15.96:0/477265052 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd26c103480 0x7fd26c1996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 >> 172.21.15.96:0/477265052 conn(0x7fd26c079bb0 msgr2=0x7fd26c1017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 shutdown_connections 2024-04-25T20:55:11.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:11.140+0000 7fd2730a8700 1 -- 172.21.15.96:0/477265052 wait complete. 2024-04-25T20:55:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:11 smithi096 ceph-mon[29288]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T20:55:11.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:11 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/477265052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:11.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:11 smithi097 ceph-mon[32667]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T20:55:11.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:11 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/477265052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:11.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:11 smithi139 ceph-mon[33027]: osdmap e93: 12 total, 12 up, 12 in 2024-04-25T20:55:11.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:11 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/477265052' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:11.837 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T20:55:11.837 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:55:12.076 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:12 smithi096 ceph-mon[29288]: pgmap v324: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-25T20:55:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:12 smithi097 ceph-mon[32667]: pgmap v324: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-25T20:55:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:12 smithi139 ceph-mon[33027]: pgmap v324: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-04-25T20:55:13.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.701+0000 7f56f0a5a700 1 -- 172.21.15.96:0/595146293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06b190 msgr2=0x7f56ec06b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:13.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.701+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/595146293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06b190 0x7f56ec06b5f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009990 tx=0x7f56e0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.701+0000 7f56f0a5a700 1 -- 172.21.15.96:0/595146293 shutdown_connections 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.701+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/595146293 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ec06bb30 0x7f56ec10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/595146293 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec108a30 0x7f56ec108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/595146293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06b190 0x7f56ec06b5f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- 172.21.15.96:0/595146293 >> 172.21.15.96:0/595146293 conn(0x7f56ec078910 msgr2=0x7f56ec078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- 172.21.15.96:0/595146293 shutdown_connections 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- 172.21.15.96:0/595146293 wait complete. 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 Processor -- start 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- start start 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec06b190 0x7f56ec199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 0x7f56ec1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ec108a30 0x7f56ec19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec112bd0 con 0x7f56ec06bb30 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec112a50 con 0x7f56ec108a30 2024-04-25T20:55:13.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.702+0000 7f56f0a5a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec112d50 con 0x7f56ec06b190 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 0x7f56ec1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 0x7f56ec1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55384/0 (socket says 172.21.15.96:55384) 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 -- 172.21.15.96:0/1974378064 learned_addr learned my addr 172.21.15.96:0/1974378064 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56ea59c700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec06b190 0x7f56ec199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 -- 172.21.15.96:0/1974378064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec06b190 msgr2=0x7f56ec199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec06b190 0x7f56ec199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 -- 172.21.15.96:0/1974378064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ec108a30 msgr2=0x7f56ec19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ec108a30 0x7f56ec19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 -- 172.21.15.96:0/1974378064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec19e170 con 0x7f56ec06bb30 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56ea59c700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec06b190 0x7f56ec199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56e9d9b700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 0x7f56ec1996a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000c00 tx=0x7f56e00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0027070 con 0x7f56ec06bb30 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e0003fa0 con 0x7f56ec06bb30 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec19e400 con 0x7f56ec06bb30 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0022db0 con 0x7f56ec06bb30 2024-04-25T20:55:13.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.703+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1a5ce0 con 0x7f56ec06bb30 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.704+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec104170 con 0x7f56ec06bb30 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.705+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f56e002a070 con 0x7f56ec06bb30 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.706+0000 7f56db7fe700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f56d40a3cd0 0x7f56d40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.706+0000 7f56ea59c700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f56d40a3cd0 0x7f56d40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.706+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f56e00d1580 con 0x7f56ec06bb30 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.706+0000 7f56ea59c700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f56d40a3cd0 0x7f56d40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f56dc005e10 tx=0x7f56dc005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:13.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.710+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56e0013070 con 0x7f56ec06bb30 2024-04-25T20:55:13.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.904+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56ec062e80 con 0x7f56d40a3cd0 2024-04-25T20:55:13.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.912+0000 7f56db7fe700 1 -- 172.21.15.96:0/1974378064 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258713 (secure 0 0 0) 0x7f56ec062e80 con 0x7f56d40a3cd0 2024-04-25T20:55:13.917 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f56d40a3cd0 msgr2=0x7f56d40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f56d40a3cd0 0x7f56d40a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f56dc005e10 tx=0x7f56dc005d80 comp rx=0 tx=0).stop 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 msgr2=0x7f56ec1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 0x7f56ec1996a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000c00 tx=0x7f56e00047b0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 shutdown_connections 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f56d40a3cd0 0x7f56d40a6190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.919 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56ec108a30 0x7f56ec19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56ec06b190 0x7f56ec199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 --2- 172.21.15.96:0/1974378064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56ec06bb30 0x7f56ec1996a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:13.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 >> 172.21.15.96:0/1974378064 conn(0x7f56ec078910 msgr2=0x7f56ec0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:13.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 shutdown_connections 2024-04-25T20:55:13.920 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:13.916+0000 7f56f0a5a700 1 -- 172.21.15.96:0/1974378064 wait complete. 2024-04-25T20:55:13.920 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:55:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:14 smithi097 ceph-mon[32667]: pgmap v325: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T20:55:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:14 smithi139 ceph-mon[33027]: pgmap v325: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T20:55:14.672 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:14 smithi096 ceph-mon[29288]: pgmap v325: 97 pgs: 97 active+clean; 630 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 6 op/s 2024-04-25T20:55:14.699 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-04-25T20:55:13.246589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":645298,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":303,"num_read_kb":260,"num_write":341,"num_write_kb":3038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":294,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379700,"kb_used_data":13852,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562124,"statfs":{"total":1151940427776,"available":1151551614976,"internally_reserved":0,"allocated":14184448,"data_stored":7806127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8611,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":42,"num_read_kb":43,"num_write":13,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001943"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.405073+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:54:31.405073+0000","last_peered":"2024-04-25T20:54:31.405073+0000","last_clean":"2024-04-25T20:54:31.405073+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:54:31.405073+0000","last_undegraded":"2024-04-25T20:54:31.405073+0000","last_fullsized":"2024-04-25T20:54:31.405073+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406648+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:54:31.406648+0000","last_peered":"2024-04-25T20:54:31.406648+0000","last_clean":"2024-04-25T20:54:31.406648+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:54:31.406648+0000","last_undegraded":"2024-04-25T20:54:31.406648+0000","last_fullsized":"2024-04-25T20:54:31.406648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406512+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:54:31.406512+0000","last_peered":"2024-04-25T20:54:31.406512+0000","last_clean":"2024-04-25T20:54:31.406512+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:54:31.406512+0000","last_undegraded":"2024-04-25T20:54:31.406512+0000","last_fullsized":"2024-04-25T20:54:31.406512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.325736+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:55:09.325736+0000","last_peered":"2024-04-25T20:55:09.325736+0000","last_clean":"2024-04-25T20:55:09.325736+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:55:09.325736+0000","last_undegraded":"2024-04-25T20:55:09.325736+0000","last_fullsized":"2024-04-25T20:55:09.325736+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331880+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:55:10.331880+0000","last_peered":"2024-04-25T20:55:10.331880+0000","last_clean":"2024-04-25T20:55:10.331880+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:55:10.331880+0000","last_undegraded":"2024-04-25T20:55:10.331880+0000","last_fullsized":"2024-04-25T20:55:10.331880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.404004+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:54:31.404004+0000","last_peered":"2024-04-25T20:54:31.404004+0000","last_clean":"2024-04-25T20:54:31.404004+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:54:31.404004+0000","last_undegraded":"2024-04-25T20:54:31.404004+0000","last_fullsized":"2024-04-25T20:54:31.404004+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400134+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:55:10.400134+0000","last_peered":"2024-04-25T20:55:10.400134+0000","last_clean":"2024-04-25T20:55:10.400134+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:55:10.400134+0000","last_undegraded":"2024-04-25T20:55:10.400134+0000","last_fullsized":"2024-04-25T20:55:10.400134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327537+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:55:09.327537+0000","last_peered":"2024-04-25T20:55:09.327537+0000","last_clean":"2024-04-25T20:55:09.327537+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:55:09.327537+0000","last_undegraded":"2024-04-25T20:55:09.327537+0000","last_fullsized":"2024-04-25T20:55:09.327537+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400633+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:55:10.400633+0000","last_peered":"2024-04-25T20:55:10.400633+0000","last_clean":"2024-04-25T20:55:10.400633+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:55:10.400633+0000","last_undegraded":"2024-04-25T20:55:10.400633+0000","last_fullsized":"2024-04-25T20:55:10.400633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.403402+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:54:31.403402+0000","last_peered":"2024-04-25T20:54:31.403402+0000","last_clean":"2024-04-25T20:54:31.403402+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:54:31.403402+0000","last_undegraded":"2024-04-25T20:54:31.403402+0000","last_fullsized":"2024-04-25T20:54:31.403402+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400845+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:55:10.400845+0000","last_peered":"2024-04-25T20:55:10.400845+0000","last_clean":"2024-04-25T20:55:10.400845+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:55:10.400845+0000","last_undegraded":"2024-04-25T20:55:10.400845+0000","last_fullsized":"2024-04-25T20:55:10.400845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.325610+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:55:09.325610+0000","last_peered":"2024-04-25T20:55:09.325610+0000","last_clean":"2024-04-25T20:55:09.325610+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:55:09.325610+0000","last_undegraded":"2024-04-25T20:55:09.325610+0000","last_fullsized":"2024-04-25T20:55:09.325610+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332732+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:55:10.332732+0000","last_peered":"2024-04-25T20:55:10.332732+0000","last_clean":"2024-04-25T20:55:10.332732+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:55:10.332732+0000","last_undegraded":"2024-04-25T20:55:10.332732+0000","last_fullsized":"2024-04-25T20:55:10.332732+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406631+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:54:31.406631+0000","last_peered":"2024-04-25T20:54:31.406631+0000","last_clean":"2024-04-25T20:54:31.406631+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:54:31.406631+0000","last_undegraded":"2024-04-25T20:54:31.406631+0000","last_fullsized":"2024-04-25T20:54:31.406631+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406157+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:54:31.406157+0000","last_peered":"2024-04-25T20:54:31.406157+0000","last_clean":"2024-04-25T20:54:31.406157+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:54:31.406157+0000","last_undegraded":"2024-04-25T20:54:31.406157+0000","last_fullsized":"2024-04-25T20:54:31.406157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.399933+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:55:10.399933+0000","last_peered":"2024-04-25T20:55:10.399933+0000","last_clean":"2024-04-25T20:55:10.399933+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:55:10.399933+0000","last_undegraded":"2024-04-25T20:55:10.399933+0000","last_fullsized":"2024-04-25T20:55:10.399933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.405487+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:54:31.405487+0000","last_peered":"2024-04-25T20:54:31.405487+0000","last_clean":"2024-04-25T20:54:31.405487+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:54:31.405487+0000","last_undegraded":"2024-04-25T20:54:31.405487+0000","last_fullsized":"2024-04-25T20:54:31.405487+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331561+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:55:10.331561+0000","last_peered":"2024-04-25T20:55:10.331561+0000","last_clean":"2024-04-25T20:55:10.331561+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:55:10.331561+0000","last_undegraded":"2024-04-25T20:55:10.331561+0000","last_fullsized":"2024-04-25T20:55:10.331561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331453+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:55:10.331453+0000","last_peered":"2024-04-25T20:55:10.331453+0000","last_clean":"2024-04-25T20:55:10.331453+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:55:10.331453+0000","last_undegraded":"2024-04-25T20:55:10.331453+0000","last_fullsized":"2024-04-25T20:55:10.331453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331587+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:55:10.331587+0000","last_peered":"2024-04-25T20:55:10.331587+0000","last_clean":"2024-04-25T20:55:10.331587+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:55:10.331587+0000","last_undegraded":"2024-04-25T20:55:10.331587+0000","last_fullsized":"2024-04-25T20:55:10.331587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327955+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:55:09.327955+0000","last_peered":"2024-04-25T20:55:09.327955+0000","last_clean":"2024-04-25T20:55:09.327955+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:55:09.327955+0000","last_undegraded":"2024-04-25T20:55:09.327955+0000","last_fullsized":"2024-04-25T20:55:09.327955+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.328166+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:55:09.328166+0000","last_peered":"2024-04-25T20:55:09.328166+0000","last_clean":"2024-04-25T20:55:09.328166+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:55:09.328166+0000","last_undegraded":"2024-04-25T20:55:09.328166+0000","last_fullsized":"2024-04-25T20:55:09.328166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406674+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:54:31.406674+0000","last_peered":"2024-04-25T20:54:31.406674+0000","last_clean":"2024-04-25T20:54:31.406674+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:54:31.406674+0000","last_undegraded":"2024-04-25T20:54:31.406674+0000","last_fullsized":"2024-04-25T20:54:31.406674+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332839+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:55:10.332839+0000","last_peered":"2024-04-25T20:55:10.332839+0000","last_clean":"2024-04-25T20:55:10.332839+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:55:10.332839+0000","last_undegraded":"2024-04-25T20:55:10.332839+0000","last_fullsized":"2024-04-25T20:55:10.332839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400275+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:55:10.400275+0000","last_peered":"2024-04-25T20:55:10.400275+0000","last_clean":"2024-04-25T20:55:10.400275+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:55:10.400275+0000","last_undegraded":"2024-04-25T20:55:10.400275+0000","last_fullsized":"2024-04-25T20:55:10.400275+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332221+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:55:10.332221+0000","last_peered":"2024-04-25T20:55:10.332221+0000","last_clean":"2024-04-25T20:55:10.332221+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:55:10.332221+0000","last_undegraded":"2024-04-25T20:55:10.332221+0000","last_fullsized":"2024-04-25T20:55:10.332221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325124+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:55:10.325124+0000","last_peered":"2024-04-25T20:55:10.325124+0000","last_clean":"2024-04-25T20:55:10.325124+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:55:10.325124+0000","last_undegraded":"2024-04-25T20:55:10.325124+0000","last_fullsized":"2024-04-25T20:55:10.325124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324896+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:55:10.324896+0000","last_peered":"2024-04-25T20:55:10.324896+0000","last_clean":"2024-04-25T20:55:10.324896+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:55:10.324896+0000","last_undegraded":"2024-04-25T20:55:10.324896+0000","last_fullsized":"2024-04-25T20:55:10.324896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332053+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:55:10.332053+0000","last_peered":"2024-04-25T20:55:10.332053+0000","last_clean":"2024-04-25T20:55:10.332053+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:55:10.332053+0000","last_undegraded":"2024-04-25T20:55:10.332053+0000","last_fullsized":"2024-04-25T20:55:10.332053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:44.333062+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:54:44.333062+0000","last_peered":"2024-04-25T20:54:44.333062+0000","last_clean":"2024-04-25T20:54:44.333062+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:54:44.333062+0000","last_undegraded":"2024-04-25T20:54:44.333062+0000","last_fullsized":"2024-04-25T20:54:44.333062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324968+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:55:10.324968+0000","last_peered":"2024-04-25T20:55:10.324968+0000","last_clean":"2024-04-25T20:55:10.324968+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:55:10.324968+0000","last_undegraded":"2024-04-25T20:55:10.324968+0000","last_fullsized":"2024-04-25T20:55:10.324968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323493+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:55:10.323493+0000","last_peered":"2024-04-25T20:55:10.323493+0000","last_clean":"2024-04-25T20:55:10.323493+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:55:10.323493+0000","last_undegraded":"2024-04-25T20:55:10.323493+0000","last_fullsized":"2024-04-25T20:55:10.323493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327438+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:55:09.327438+0000","last_peered":"2024-04-25T20:55:09.327438+0000","last_clean":"2024-04-25T20:55:09.327438+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:55:09.327438+0000","last_undegraded":"2024-04-25T20:55:09.327438+0000","last_fullsized":"2024-04-25T20:55:09.327438+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324604+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:55:10.324604+0000","last_peered":"2024-04-25T20:55:10.324604+0000","last_clean":"2024-04-25T20:55:10.324604+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:55:10.324604+0000","last_undegraded":"2024-04-25T20:55:10.324604+0000","last_fullsized":"2024-04-25T20:55:10.324604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331411+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:55:10.331411+0000","last_peered":"2024-04-25T20:55:10.331411+0000","last_clean":"2024-04-25T20:55:10.331411+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:55:10.331411+0000","last_undegraded":"2024-04-25T20:55:10.331411+0000","last_fullsized":"2024-04-25T20:55:10.331411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:55:06.385063+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:55:06.385063+0000","last_peered":"2024-04-25T20:55:06.385063+0000","last_clean":"2024-04-25T20:55:06.385063+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:55:06.385063+0000","last_undegraded":"2024-04-25T20:55:06.385063+0000","last_fullsized":"2024-04-25T20:55:06.385063+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400580+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:55:10.400580+0000","last_peered":"2024-04-25T20:55:10.400580+0000","last_clean":"2024-04-25T20:55:10.400580+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:55:10.400580+0000","last_undegraded":"2024-04-25T20:55:10.400580+0000","last_fullsized":"2024-04-25T20:55:10.400580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400744+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:55:10.400744+0000","last_peered":"2024-04-25T20:55:10.400744+0000","last_clean":"2024-04-25T20:55:10.400744+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:55:10.400744+0000","last_undegraded":"2024-04-25T20:55:10.400744+0000","last_fullsized":"2024-04-25T20:55:10.400744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406798+0000","last_change":"2024-04-25T20:54:29.400875+0000","last_active":"2024-04-25T20:54:31.406798+0000","last_peered":"2024-04-25T20:54:31.406798+0000","last_clean":"2024-04-25T20:54:31.406798+0000","last_became_active":"2024-04-25T20:54:29.400542+0000","last_became_peered":"2024-04-25T20:54:29.400542+0000","last_unstale":"2024-04-25T20:54:31.406798+0000","last_undegraded":"2024-04-25T20:54:31.406798+0000","last_fullsized":"2024-04-25T20:54:31.406798+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:47:08.734131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325197+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:55:10.325197+0000","last_peered":"2024-04-25T20:55:10.325197+0000","last_clean":"2024-04-25T20:55:10.325197+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:55:10.325197+0000","last_undegraded":"2024-04-25T20:55:10.325197+0000","last_fullsized":"2024-04-25T20:55:10.325197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400426+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:55:10.400426+0000","last_peered":"2024-04-25T20:55:10.400426+0000","last_clean":"2024-04-25T20:55:10.400426+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:55:10.400426+0000","last_undegraded":"2024-04-25T20:55:10.400426+0000","last_fullsized":"2024-04-25T20:55:10.400426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'25","reported_seq":97,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:55:07.387350+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:55:07.387350+0000","last_peered":"2024-04-25T20:55:07.387350+0000","last_clean":"2024-04-25T20:55:07.387350+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:55:07.387350+0000","last_undegraded":"2024-04-25T20:55:07.387350+0000","last_fullsized":"2024-04-25T20:55:07.387350+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":50695,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":51,"num_read_kb":74,"num_write":26,"num_write_kb":57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332629+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:55:10.332629+0000","last_peered":"2024-04-25T20:55:10.332629+0000","last_clean":"2024-04-25T20:55:10.332629+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:55:10.332629+0000","last_undegraded":"2024-04-25T20:55:10.332629+0000","last_fullsized":"2024-04-25T20:55:10.332629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331406+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:55:10.331406+0000","last_peered":"2024-04-25T20:55:10.331406+0000","last_clean":"2024-04-25T20:55:10.331406+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:55:10.331406+0000","last_undegraded":"2024-04-25T20:55:10.331406+0000","last_fullsized":"2024-04-25T20:55:10.331406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323434+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:55:10.323434+0000","last_peered":"2024-04-25T20:55:10.323434+0000","last_clean":"2024-04-25T20:55:10.323434+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:55:10.323434+0000","last_undegraded":"2024-04-25T20:55:10.323434+0000","last_fullsized":"2024-04-25T20:55:10.323434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324988+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:55:10.324988+0000","last_peered":"2024-04-25T20:55:10.324988+0000","last_clean":"2024-04-25T20:55:10.324988+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:55:10.324988+0000","last_undegraded":"2024-04-25T20:55:10.324988+0000","last_fullsized":"2024-04-25T20:55:10.324988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400316+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:55:10.400316+0000","last_peered":"2024-04-25T20:55:10.400316+0000","last_clean":"2024-04-25T20:55:10.400316+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:55:10.400316+0000","last_undegraded":"2024-04-25T20:55:10.400316+0000","last_fullsized":"2024-04-25T20:55:10.400316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.325274+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:55:09.325274+0000","last_peered":"2024-04-25T20:55:09.325274+0000","last_clean":"2024-04-25T20:55:09.325274+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:55:09.325274+0000","last_undegraded":"2024-04-25T20:55:09.325274+0000","last_fullsized":"2024-04-25T20:55:09.325274+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400116+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:55:10.400116+0000","last_peered":"2024-04-25T20:55:10.400116+0000","last_clean":"2024-04-25T20:55:10.400116+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:55:10.400116+0000","last_undegraded":"2024-04-25T20:55:10.400116+0000","last_fullsized":"2024-04-25T20:55:10.400116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332109+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:55:10.332109+0000","last_peered":"2024-04-25T20:55:10.332109+0000","last_clean":"2024-04-25T20:55:10.332109+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:55:10.332109+0000","last_undegraded":"2024-04-25T20:55:10.332109+0000","last_fullsized":"2024-04-25T20:55:10.332109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324690+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:55:10.324690+0000","last_peered":"2024-04-25T20:55:10.324690+0000","last_clean":"2024-04-25T20:55:10.324690+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:55:10.324690+0000","last_undegraded":"2024-04-25T20:55:10.324690+0000","last_fullsized":"2024-04-25T20:55:10.324690+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332534+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:55:10.332534+0000","last_peered":"2024-04-25T20:55:10.332534+0000","last_clean":"2024-04-25T20:55:10.332534+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:55:10.332534+0000","last_undegraded":"2024-04-25T20:55:10.332534+0000","last_fullsized":"2024-04-25T20:55:10.332534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.403928+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:54:31.403928+0000","last_peered":"2024-04-25T20:54:31.403928+0000","last_clean":"2024-04-25T20:54:31.403928+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:54:31.403928+0000","last_undegraded":"2024-04-25T20:54:31.403928+0000","last_fullsized":"2024-04-25T20:54:31.403928+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331313+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:55:10.331313+0000","last_peered":"2024-04-25T20:55:10.331313+0000","last_clean":"2024-04-25T20:55:10.331313+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:55:10.331313+0000","last_undegraded":"2024-04-25T20:55:10.331313+0000","last_fullsized":"2024-04-25T20:55:10.331313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'1","reported_seq":15,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:33.323582+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:54:33.323582+0000","last_peered":"2024-04-25T20:54:33.323582+0000","last_clean":"2024-04-25T20:54:33.323582+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:54:33.323582+0000","last_undegraded":"2024-04-25T20:54:33.323582+0000","last_fullsized":"2024-04-25T20:54:33.323582+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400439+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:55:10.400439+0000","last_peered":"2024-04-25T20:55:10.400439+0000","last_clean":"2024-04-25T20:55:10.400439+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:55:10.400439+0000","last_undegraded":"2024-04-25T20:55:10.400439+0000","last_fullsized":"2024-04-25T20:55:10.400439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.402392+0000","last_change":"2024-04-25T20:54:29.399524+0000","last_active":"2024-04-25T20:54:31.402392+0000","last_peered":"2024-04-25T20:54:31.402392+0000","last_clean":"2024-04-25T20:54:31.402392+0000","last_became_active":"2024-04-25T20:54:29.399145+0000","last_became_peered":"2024-04-25T20:54:29.399145+0000","last_unstale":"2024-04-25T20:54:31.402392+0000","last_undegraded":"2024-04-25T20:54:31.402392+0000","last_fullsized":"2024-04-25T20:54:31.402392+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:52.782950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325183+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:55:10.325183+0000","last_peered":"2024-04-25T20:55:10.325183+0000","last_clean":"2024-04-25T20:55:10.325183+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:55:10.325183+0000","last_undegraded":"2024-04-25T20:55:10.325183+0000","last_fullsized":"2024-04-25T20:55:10.325183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.390364+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:55:10.390364+0000","last_peered":"2024-04-25T20:55:10.390364+0000","last_clean":"2024-04-25T20:55:10.390364+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:55:10.390364+0000","last_undegraded":"2024-04-25T20:55:10.390364+0000","last_fullsized":"2024-04-25T20:55:10.390364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327976+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:55:09.327976+0000","last_peered":"2024-04-25T20:55:09.327976+0000","last_clean":"2024-04-25T20:55:09.327976+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:55:09.327976+0000","last_undegraded":"2024-04-25T20:55:09.327976+0000","last_fullsized":"2024-04-25T20:55:09.327976+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":28,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.325839+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:55:09.325839+0000","last_peered":"2024-04-25T20:55:09.325839+0000","last_clean":"2024-04-25T20:55:09.325839+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:55:09.325839+0000","last_undegraded":"2024-04-25T20:55:09.325839+0000","last_fullsized":"2024-04-25T20:55:09.325839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400226+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:55:10.400226+0000","last_peered":"2024-04-25T20:55:10.400226+0000","last_clean":"2024-04-25T20:55:10.400226+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:55:10.400226+0000","last_undegraded":"2024-04-25T20:55:10.400226+0000","last_fullsized":"2024-04-25T20:55:10.400226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323353+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:55:10.323353+0000","last_peered":"2024-04-25T20:55:10.323353+0000","last_clean":"2024-04-25T20:55:10.323353+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:55:10.323353+0000","last_undegraded":"2024-04-25T20:55:10.323353+0000","last_fullsized":"2024-04-25T20:55:10.323353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400305+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:55:10.400305+0000","last_peered":"2024-04-25T20:55:10.400305+0000","last_clean":"2024-04-25T20:55:10.400305+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:55:10.400305+0000","last_undegraded":"2024-04-25T20:55:10.400305+0000","last_fullsized":"2024-04-25T20:55:10.400305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":24,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:43.326411+0000","last_change":"2024-04-25T20:54:26.549858+0000","last_active":"2024-04-25T20:54:43.326411+0000","last_peered":"2024-04-25T20:54:43.326411+0000","last_clean":"2024-04-25T20:54:43.326411+0000","last_became_active":"2024-04-25T20:54:26.549446+0000","last_became_peered":"2024-04-25T20:54:26.549446+0000","last_unstale":"2024-04-25T20:54:43.326411+0000","last_undegraded":"2024-04-25T20:54:43.326411+0000","last_fullsized":"2024-04-25T20:54:43.326411+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:08:39.960990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323526+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:55:10.323526+0000","last_peered":"2024-04-25T20:55:10.323526+0000","last_clean":"2024-04-25T20:55:10.323526+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:55:10.323526+0000","last_undegraded":"2024-04-25T20:55:10.323526+0000","last_fullsized":"2024-04-25T20:55:10.323526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331236+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:55:10.331236+0000","last_peered":"2024-04-25T20:55:10.331236+0000","last_clean":"2024-04-25T20:55:10.331236+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:55:10.331236+0000","last_undegraded":"2024-04-25T20:55:10.331236+0000","last_fullsized":"2024-04-25T20:55:10.331236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.328119+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:55:09.328119+0000","last_peered":"2024-04-25T20:55:09.328119+0000","last_clean":"2024-04-25T20:55:09.328119+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:55:09.328119+0000","last_undegraded":"2024-04-25T20:55:09.328119+0000","last_fullsized":"2024-04-25T20:55:09.328119+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.325475+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:55:09.325475+0000","last_peered":"2024-04-25T20:55:09.325475+0000","last_clean":"2024-04-25T20:55:09.325475+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:55:09.325475+0000","last_undegraded":"2024-04-25T20:55:09.325475+0000","last_fullsized":"2024-04-25T20:55:09.325475+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406808+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:54:31.406808+0000","last_peered":"2024-04-25T20:54:31.406808+0000","last_clean":"2024-04-25T20:54:31.406808+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:54:31.406808+0000","last_undegraded":"2024-04-25T20:54:31.406808+0000","last_fullsized":"2024-04-25T20:54:31.406808+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.399951+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:55:10.399951+0000","last_peered":"2024-04-25T20:55:10.399951+0000","last_clean":"2024-04-25T20:55:10.399951+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:55:10.399951+0000","last_undegraded":"2024-04-25T20:55:10.399951+0000","last_fullsized":"2024-04-25T20:55:10.399951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331875+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:55:10.331875+0000","last_peered":"2024-04-25T20:55:10.331875+0000","last_clean":"2024-04-25T20:55:10.331875+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:55:10.331875+0000","last_undegraded":"2024-04-25T20:55:10.331875+0000","last_fullsized":"2024-04-25T20:55:10.331875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327666+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:55:09.327666+0000","last_peered":"2024-04-25T20:55:09.327666+0000","last_clean":"2024-04-25T20:55:09.327666+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:55:09.327666+0000","last_undegraded":"2024-04-25T20:55:09.327666+0000","last_fullsized":"2024-04-25T20:55:09.327666+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.403365+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:54:31.403365+0000","last_peered":"2024-04-25T20:54:31.403365+0000","last_clean":"2024-04-25T20:54:31.403365+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:54:31.403365+0000","last_undegraded":"2024-04-25T20:54:31.403365+0000","last_fullsized":"2024-04-25T20:54:31.403365+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.390684+0000","last_change":"2024-04-25T20:54:26.360454+0000","last_active":"2024-04-25T20:54:31.390684+0000","last_peered":"2024-04-25T20:54:31.390684+0000","last_clean":"2024-04-25T20:54:31.390684+0000","last_became_active":"2024-04-25T20:54:26.359832+0000","last_became_peered":"2024-04-25T20:54:26.359832+0000","last_unstale":"2024-04-25T20:54:31.390684+0000","last_undegraded":"2024-04-25T20:54:31.390684+0000","last_fullsized":"2024-04-25T20:54:31.390684+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:08:13.692252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327930+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:55:09.327930+0000","last_peered":"2024-04-25T20:55:09.327930+0000","last_clean":"2024-04-25T20:55:09.327930+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:55:09.327930+0000","last_undegraded":"2024-04-25T20:55:09.327930+0000","last_fullsized":"2024-04-25T20:55:09.327930+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331929+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:55:10.331929+0000","last_peered":"2024-04-25T20:55:10.331929+0000","last_clean":"2024-04-25T20:55:10.331929+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:55:10.331929+0000","last_undegraded":"2024-04-25T20:55:10.331929+0000","last_fullsized":"2024-04-25T20:55:10.331929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400510+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:55:10.400510+0000","last_peered":"2024-04-25T20:55:10.400510+0000","last_clean":"2024-04-25T20:55:10.400510+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:55:10.400510+0000","last_undegraded":"2024-04-25T20:55:10.400510+0000","last_fullsized":"2024-04-25T20:55:10.400510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331766+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:55:10.331766+0000","last_peered":"2024-04-25T20:55:10.331766+0000","last_clean":"2024-04-25T20:55:10.331766+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:55:10.331766+0000","last_undegraded":"2024-04-25T20:55:10.331766+0000","last_fullsized":"2024-04-25T20:55:10.331766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400938+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:55:10.400938+0000","last_peered":"2024-04-25T20:55:10.400938+0000","last_clean":"2024-04-25T20:55:10.400938+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:55:10.400938+0000","last_undegraded":"2024-04-25T20:55:10.400938+0000","last_fullsized":"2024-04-25T20:55:10.400938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325055+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:55:10.325055+0000","last_peered":"2024-04-25T20:55:10.325055+0000","last_clean":"2024-04-25T20:55:10.325055+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:55:10.325055+0000","last_undegraded":"2024-04-25T20:55:10.325055+0000","last_fullsized":"2024-04-25T20:55:10.325055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400705+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:55:10.400705+0000","last_peered":"2024-04-25T20:55:10.400705+0000","last_clean":"2024-04-25T20:55:10.400705+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:55:10.400705+0000","last_undegraded":"2024-04-25T20:55:10.400705+0000","last_fullsized":"2024-04-25T20:55:10.400705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323575+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:55:10.323575+0000","last_peered":"2024-04-25T20:55:10.323575+0000","last_clean":"2024-04-25T20:55:10.323575+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:55:10.323575+0000","last_undegraded":"2024-04-25T20:55:10.323575+0000","last_fullsized":"2024-04-25T20:55:10.323575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400910+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:55:10.400910+0000","last_peered":"2024-04-25T20:55:10.400910+0000","last_clean":"2024-04-25T20:55:10.400910+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:55:10.400910+0000","last_undegraded":"2024-04-25T20:55:10.400910+0000","last_fullsized":"2024-04-25T20:55:10.400910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.403562+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:54:31.403562+0000","last_peered":"2024-04-25T20:54:31.403562+0000","last_clean":"2024-04-25T20:54:31.403562+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:54:31.403562+0000","last_undegraded":"2024-04-25T20:54:31.403562+0000","last_fullsized":"2024-04-25T20:54:31.403562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331088+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:55:10.331088+0000","last_peered":"2024-04-25T20:55:10.331088+0000","last_clean":"2024-04-25T20:55:10.331088+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:55:10.331088+0000","last_undegraded":"2024-04-25T20:55:10.331088+0000","last_fullsized":"2024-04-25T20:55:10.331088+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330927+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:55:10.330927+0000","last_peered":"2024-04-25T20:55:10.330927+0000","last_clean":"2024-04-25T20:55:10.330927+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:55:10.330927+0000","last_undegraded":"2024-04-25T20:55:10.330927+0000","last_fullsized":"2024-04-25T20:55:10.330927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.406648+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:54:31.406648+0000","last_peered":"2024-04-25T20:54:31.406648+0000","last_clean":"2024-04-25T20:54:31.406648+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:54:31.406648+0000","last_undegraded":"2024-04-25T20:54:31.406648+0000","last_fullsized":"2024-04-25T20:54:31.406648+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332688+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:55:10.332688+0000","last_peered":"2024-04-25T20:55:10.332688+0000","last_clean":"2024-04-25T20:55:10.332688+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:55:10.332688+0000","last_undegraded":"2024-04-25T20:55:10.332688+0000","last_fullsized":"2024-04-25T20:55:10.332688+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.404432+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:54:31.404432+0000","last_peered":"2024-04-25T20:54:31.404432+0000","last_clean":"2024-04-25T20:54:31.404432+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:54:31.404432+0000","last_undegraded":"2024-04-25T20:54:31.404432+0000","last_fullsized":"2024-04-25T20:54:31.404432+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331638+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:55:10.331638+0000","last_peered":"2024-04-25T20:55:10.331638+0000","last_clean":"2024-04-25T20:55:10.331638+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:55:10.331638+0000","last_undegraded":"2024-04-25T20:55:10.331638+0000","last_fullsized":"2024-04-25T20:55:10.331638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.328023+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:55:09.328023+0000","last_peered":"2024-04-25T20:55:09.328023+0000","last_clean":"2024-04-25T20:55:09.328023+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:55:09.328023+0000","last_undegraded":"2024-04-25T20:55:09.328023+0000","last_fullsized":"2024-04-25T20:55:09.328023+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324727+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:55:10.324727+0000","last_peered":"2024-04-25T20:55:10.324727+0000","last_clean":"2024-04-25T20:55:10.324727+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:55:10.324727+0000","last_undegraded":"2024-04-25T20:55:10.324727+0000","last_fullsized":"2024-04-25T20:55:10.324727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-04-25T20:55:09.327658+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:55:09.327658+0000","last_peered":"2024-04-25T20:55:09.327658+0000","last_clean":"2024-04-25T20:55:09.327658+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:55:09.327658+0000","last_undegraded":"2024-04-25T20:55:09.327658+0000","last_fullsized":"2024-04-25T20:55:09.327658+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400164+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:55:10.400164+0000","last_peered":"2024-04-25T20:55:10.400164+0000","last_clean":"2024-04-25T20:55:10.400164+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:55:10.400164+0000","last_undegraded":"2024-04-25T20:55:10.400164+0000","last_fullsized":"2024-04-25T20:55:10.400164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323589+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:55:10.323589+0000","last_peered":"2024-04-25T20:55:10.323589+0000","last_clean":"2024-04-25T20:55:10.323589+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:55:10.323589+0000","last_undegraded":"2024-04-25T20:55:10.323589+0000","last_fullsized":"2024-04-25T20:55:10.323589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-25T20:54:31.403297+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:54:31.403297+0000","last_peered":"2024-04-25T20:54:31.403297+0000","last_clean":"2024-04-25T20:54:31.403297+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:54:31.403297+0000","last_undegraded":"2024-04-25T20:54:31.403297+0000","last_fullsized":"2024-04-25T20:54:31.403297+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":54696,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":167,"num_read_kb":141,"num_write":158,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":397312,"data_stored":174856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285593,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":478283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547230,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32812,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712340,"statfs":{"total":95995035648,"available":95961436160,"internally_reserved":0,"allocated":1675264,"data_stored":1120459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":495431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005095,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":495329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1609728,"data_stored":1078896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":494820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":550703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":479179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":552444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":486746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888123,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1007616,"data_stored":486872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1086965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":52273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:55:14.704 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:55:14.940 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:15 smithi097 ceph-mon[32667]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:55:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:15 smithi139 ceph-mon[33027]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:55:15.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:15 smithi096 ceph-mon[29288]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:55:16.421 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:16 smithi096 ceph-mon[29288]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:16 smithi097 ceph-mon[32667]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:16 smithi139 ceph-mon[33027]: pgmap v326: 97 pgs: 97 active+clean; 635 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.5 KiB/s wr, 8 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 -- 172.21.15.96:0/841918474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c075590 msgr2=0x7f040c10ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 --2- 172.21.15.96:0/841918474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c075590 0x7f040c10ef70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f03fc009990 tx=0x7f03fc009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 -- 172.21.15.96:0/841918474 shutdown_connections 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 --2- 172.21.15.96:0/841918474 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c10f4b0 0x7f040c1118a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 --2- 172.21.15.96:0/841918474 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f040c074c70 0x7f040c075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 --2- 172.21.15.96:0/841918474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c075590 0x7f040c10ef70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 -- 172.21.15.96:0/841918474 >> 172.21.15.96:0/841918474 conn(0x7f040c079c00 msgr2=0x7f040c07a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 -- 172.21.15.96:0/841918474 shutdown_connections 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.665+0000 7f0413358700 1 -- 172.21.15.96:0/841918474 wait complete. 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 Processor -- start 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 -- start start 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f040c074c70 0x7f040c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c075590 0x7f040c19b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 0x7f040c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f040c114de0 con 0x7f040c10f4b0 2024-04-25T20:55:16.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f040c114c60 con 0x7f040c075590 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f0413358700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f040c114f60 con 0x7f040c074c70 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f04118f5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 0x7f040c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f04118f5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 0x7f040c19fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33742/0 (socket says 172.21.15.96:33742) 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.666+0000 7f04118f5700 1 -- 172.21.15.96:0/686908683 learned_addr learned my addr 172.21.15.96:0/686908683 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04108f3700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c075590 0x7f040c19b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04110f4700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f040c074c70 0x7f040c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04118f5700 1 -- 172.21.15.96:0/686908683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f040c074c70 msgr2=0x7f040c19b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04118f5700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f040c074c70 0x7f040c19b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04118f5700 1 -- 172.21.15.96:0/686908683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c075590 msgr2=0x7f040c19b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04118f5700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c075590 0x7f040c19b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04118f5700 1 -- 172.21.15.96:0/686908683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040c1a0320 con 0x7f040c10f4b0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04108f3700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c075590 0x7f040c19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04118f5700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 0x7f040c19fbe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0404007ee0 tx=0x7f040400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040400dbd0 con 0x7f040c10f4b0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040400dd70 con 0x7f040c10f4b0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040c1a0610 con 0x7f040c10f4b0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0404005990 con 0x7f040c10f4b0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.667+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f040c1a7e90 con 0x7f040c10f4b0 2024-04-25T20:55:16.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.668+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f040c069730 con 0x7f040c10f4b0 2024-04-25T20:55:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.669+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f040401a020 con 0x7f040c10f4b0 2024-04-25T20:55:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.670+0000 7f04027fc700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03f80a3cd0 0x7f03f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.670+0000 7f04110f4700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03f80a3cd0 0x7f03f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.670+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f04040cfc70 con 0x7f040c10f4b0 2024-04-25T20:55:16.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.670+0000 7f04110f4700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03f80a3cd0 0x7f03f80a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0408005950 tx=0x7f04080058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:16.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.674+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0404010070 con 0x7f040c10f4b0 2024-04-25T20:55:16.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.868+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f040c062e80 con 0x7f03f80a3cd0 2024-04-25T20:55:16.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.875+0000 7f04027fc700 1 -- 172.21.15.96:0/686908683 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258730 (secure 0 0 0) 0x7f040c062e80 con 0x7f03f80a3cd0 2024-04-25T20:55:16.880 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:16.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03f80a3cd0 msgr2=0x7f03f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:16.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03f80a3cd0 0x7f03f80a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0408005950 tx=0x7f04080058e0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 msgr2=0x7f040c19fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 0x7f040c19fbe0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0404007ee0 tx=0x7f040400d3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 shutdown_connections 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f03f80a3cd0 0x7f03f80a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f040c10f4b0 0x7f040c19fbe0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f040c074c70 0x7f040c19b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 --2- 172.21.15.96:0/686908683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f040c075590 0x7f040c19b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 >> 172.21.15.96:0/686908683 conn(0x7f040c079c00 msgr2=0x7f040c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 shutdown_connections 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:16.878+0000 7f0413358700 1 -- 172.21.15.96:0/686908683 wait complete. 2024-04-25T20:55:16.883 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:55:17.665 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-04-25T20:55:15.247196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650693,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":323,"num_read_kb":275,"num_write":344,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379828,"kb_used_data":13956,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561996,"statfs":{"total":1151940427776,"available":1151551483904,"internally_reserved":0,"allocated":14290944,"data_stored":7879050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":374453564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13889,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":57,"num_read_kb":58,"num_write":15,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002034"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.328113+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:55:10.328113+0000","last_peered":"2024-04-25T20:55:10.328113+0000","last_clean":"2024-04-25T20:55:10.328113+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:55:10.328113+0000","last_undegraded":"2024-04-25T20:55:10.328113+0000","last_fullsized":"2024-04-25T20:55:10.328113+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326293+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:55:10.326293+0000","last_peered":"2024-04-25T20:55:10.326293+0000","last_clean":"2024-04-25T20:55:10.326293+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:55:10.326293+0000","last_undegraded":"2024-04-25T20:55:10.326293+0000","last_fullsized":"2024-04-25T20:55:10.326293+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326334+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:55:10.326334+0000","last_peered":"2024-04-25T20:55:10.326334+0000","last_clean":"2024-04-25T20:55:10.326334+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:55:10.326334+0000","last_undegraded":"2024-04-25T20:55:10.326334+0000","last_fullsized":"2024-04-25T20:55:10.326334+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324692+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:55:10.324692+0000","last_peered":"2024-04-25T20:55:10.324692+0000","last_clean":"2024-04-25T20:55:10.324692+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:55:10.324692+0000","last_undegraded":"2024-04-25T20:55:10.324692+0000","last_fullsized":"2024-04-25T20:55:10.324692+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331880+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:55:10.331880+0000","last_peered":"2024-04-25T20:55:10.331880+0000","last_clean":"2024-04-25T20:55:10.331880+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:55:10.331880+0000","last_undegraded":"2024-04-25T20:55:10.331880+0000","last_fullsized":"2024-04-25T20:55:10.331880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326693+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:55:10.326693+0000","last_peered":"2024-04-25T20:55:10.326693+0000","last_clean":"2024-04-25T20:55:10.326693+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:55:10.326693+0000","last_undegraded":"2024-04-25T20:55:10.326693+0000","last_fullsized":"2024-04-25T20:55:10.326693+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400134+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:55:10.400134+0000","last_peered":"2024-04-25T20:55:10.400134+0000","last_clean":"2024-04-25T20:55:10.400134+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:55:10.400134+0000","last_undegraded":"2024-04-25T20:55:10.400134+0000","last_fullsized":"2024-04-25T20:55:10.400134+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330510+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:55:10.330510+0000","last_peered":"2024-04-25T20:55:10.330510+0000","last_clean":"2024-04-25T20:55:10.330510+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:55:10.330510+0000","last_undegraded":"2024-04-25T20:55:10.330510+0000","last_fullsized":"2024-04-25T20:55:10.330510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400633+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:55:10.400633+0000","last_peered":"2024-04-25T20:55:10.400633+0000","last_clean":"2024-04-25T20:55:10.400633+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:55:10.400633+0000","last_undegraded":"2024-04-25T20:55:10.400633+0000","last_fullsized":"2024-04-25T20:55:10.400633+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326960+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:55:10.326960+0000","last_peered":"2024-04-25T20:55:10.326960+0000","last_clean":"2024-04-25T20:55:10.326960+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:55:10.326960+0000","last_undegraded":"2024-04-25T20:55:10.326960+0000","last_fullsized":"2024-04-25T20:55:10.326960+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400845+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:55:10.400845+0000","last_peered":"2024-04-25T20:55:10.400845+0000","last_clean":"2024-04-25T20:55:10.400845+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:55:10.400845+0000","last_undegraded":"2024-04-25T20:55:10.400845+0000","last_fullsized":"2024-04-25T20:55:10.400845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324592+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:55:10.324592+0000","last_peered":"2024-04-25T20:55:10.324592+0000","last_clean":"2024-04-25T20:55:10.324592+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:55:10.324592+0000","last_undegraded":"2024-04-25T20:55:10.324592+0000","last_fullsized":"2024-04-25T20:55:10.324592+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332732+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:55:10.332732+0000","last_peered":"2024-04-25T20:55:10.332732+0000","last_clean":"2024-04-25T20:55:10.332732+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:55:10.332732+0000","last_undegraded":"2024-04-25T20:55:10.332732+0000","last_fullsized":"2024-04-25T20:55:10.332732+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326237+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:55:10.326237+0000","last_peered":"2024-04-25T20:55:10.326237+0000","last_clean":"2024-04-25T20:55:10.326237+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:55:10.326237+0000","last_undegraded":"2024-04-25T20:55:10.326237+0000","last_fullsized":"2024-04-25T20:55:10.326237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326141+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:55:10.326141+0000","last_peered":"2024-04-25T20:55:10.326141+0000","last_clean":"2024-04-25T20:55:10.326141+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:55:10.326141+0000","last_undegraded":"2024-04-25T20:55:10.326141+0000","last_fullsized":"2024-04-25T20:55:10.326141+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.399933+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:55:10.399933+0000","last_peered":"2024-04-25T20:55:10.399933+0000","last_clean":"2024-04-25T20:55:10.399933+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:55:10.399933+0000","last_undegraded":"2024-04-25T20:55:10.399933+0000","last_fullsized":"2024-04-25T20:55:10.399933+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.328317+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:55:10.328317+0000","last_peered":"2024-04-25T20:55:10.328317+0000","last_clean":"2024-04-25T20:55:10.328317+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:55:10.328317+0000","last_undegraded":"2024-04-25T20:55:10.328317+0000","last_fullsized":"2024-04-25T20:55:10.328317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331561+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:55:10.331561+0000","last_peered":"2024-04-25T20:55:10.331561+0000","last_clean":"2024-04-25T20:55:10.331561+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:55:10.331561+0000","last_undegraded":"2024-04-25T20:55:10.331561+0000","last_fullsized":"2024-04-25T20:55:10.331561+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331453+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:55:10.331453+0000","last_peered":"2024-04-25T20:55:10.331453+0000","last_clean":"2024-04-25T20:55:10.331453+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:55:10.331453+0000","last_undegraded":"2024-04-25T20:55:10.331453+0000","last_fullsized":"2024-04-25T20:55:10.331453+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331587+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:55:10.331587+0000","last_peered":"2024-04-25T20:55:10.331587+0000","last_clean":"2024-04-25T20:55:10.331587+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:55:10.331587+0000","last_undegraded":"2024-04-25T20:55:10.331587+0000","last_fullsized":"2024-04-25T20:55:10.331587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330833+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:55:10.330833+0000","last_peered":"2024-04-25T20:55:10.330833+0000","last_clean":"2024-04-25T20:55:10.330833+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:55:10.330833+0000","last_undegraded":"2024-04-25T20:55:10.330833+0000","last_fullsized":"2024-04-25T20:55:10.330833+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331113+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:55:10.331113+0000","last_peered":"2024-04-25T20:55:10.331113+0000","last_clean":"2024-04-25T20:55:10.331113+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:55:10.331113+0000","last_undegraded":"2024-04-25T20:55:10.331113+0000","last_fullsized":"2024-04-25T20:55:10.331113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326860+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:55:10.326860+0000","last_peered":"2024-04-25T20:55:10.326860+0000","last_clean":"2024-04-25T20:55:10.326860+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:55:10.326860+0000","last_undegraded":"2024-04-25T20:55:10.326860+0000","last_fullsized":"2024-04-25T20:55:10.326860+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332839+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:55:10.332839+0000","last_peered":"2024-04-25T20:55:10.332839+0000","last_clean":"2024-04-25T20:55:10.332839+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:55:10.332839+0000","last_undegraded":"2024-04-25T20:55:10.332839+0000","last_fullsized":"2024-04-25T20:55:10.332839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400275+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:55:10.400275+0000","last_peered":"2024-04-25T20:55:10.400275+0000","last_clean":"2024-04-25T20:55:10.400275+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:55:10.400275+0000","last_undegraded":"2024-04-25T20:55:10.400275+0000","last_fullsized":"2024-04-25T20:55:10.400275+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332221+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:55:10.332221+0000","last_peered":"2024-04-25T20:55:10.332221+0000","last_clean":"2024-04-25T20:55:10.332221+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:55:10.332221+0000","last_undegraded":"2024-04-25T20:55:10.332221+0000","last_fullsized":"2024-04-25T20:55:10.332221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325124+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:55:10.325124+0000","last_peered":"2024-04-25T20:55:10.325124+0000","last_clean":"2024-04-25T20:55:10.325124+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:55:10.325124+0000","last_undegraded":"2024-04-25T20:55:10.325124+0000","last_fullsized":"2024-04-25T20:55:10.325124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'6","reported_seq":60,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324896+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:55:10.324896+0000","last_peered":"2024-04-25T20:55:10.324896+0000","last_clean":"2024-04-25T20:55:10.324896+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:55:10.324896+0000","last_undegraded":"2024-04-25T20:55:10.324896+0000","last_fullsized":"2024-04-25T20:55:10.324896+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332053+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:55:10.332053+0000","last_peered":"2024-04-25T20:55:10.332053+0000","last_clean":"2024-04-25T20:55:10.332053+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:55:10.332053+0000","last_undegraded":"2024-04-25T20:55:10.332053+0000","last_fullsized":"2024-04-25T20:55:10.332053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326638+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:55:10.326638+0000","last_peered":"2024-04-25T20:55:10.326638+0000","last_clean":"2024-04-25T20:55:10.326638+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:55:10.326638+0000","last_undegraded":"2024-04-25T20:55:10.326638+0000","last_fullsized":"2024-04-25T20:55:10.326638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324968+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:55:10.324968+0000","last_peered":"2024-04-25T20:55:10.324968+0000","last_clean":"2024-04-25T20:55:10.324968+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:55:10.324968+0000","last_undegraded":"2024-04-25T20:55:10.324968+0000","last_fullsized":"2024-04-25T20:55:10.324968+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323493+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:55:10.323493+0000","last_peered":"2024-04-25T20:55:10.323493+0000","last_clean":"2024-04-25T20:55:10.323493+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:55:10.323493+0000","last_undegraded":"2024-04-25T20:55:10.323493+0000","last_fullsized":"2024-04-25T20:55:10.323493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330065+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:55:10.330065+0000","last_peered":"2024-04-25T20:55:10.330065+0000","last_clean":"2024-04-25T20:55:10.330065+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:55:10.330065+0000","last_undegraded":"2024-04-25T20:55:10.330065+0000","last_fullsized":"2024-04-25T20:55:10.330065+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324604+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:55:10.324604+0000","last_peered":"2024-04-25T20:55:10.324604+0000","last_clean":"2024-04-25T20:55:10.324604+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:55:10.324604+0000","last_undegraded":"2024-04-25T20:55:10.324604+0000","last_fullsized":"2024-04-25T20:55:10.324604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331411+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:55:10.331411+0000","last_peered":"2024-04-25T20:55:10.331411+0000","last_clean":"2024-04-25T20:55:10.331411+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:55:10.331411+0000","last_undegraded":"2024-04-25T20:55:10.331411+0000","last_fullsized":"2024-04-25T20:55:10.331411+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'3","reported_seq":55,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:11.392264+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:55:11.392264+0000","last_peered":"2024-04-25T20:55:11.392264+0000","last_clean":"2024-04-25T20:55:11.392264+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:55:11.392264+0000","last_undegraded":"2024-04-25T20:55:11.392264+0000","last_fullsized":"2024-04-25T20:55:11.392264+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400580+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:55:10.400580+0000","last_peered":"2024-04-25T20:55:10.400580+0000","last_clean":"2024-04-25T20:55:10.400580+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:55:10.400580+0000","last_undegraded":"2024-04-25T20:55:10.400580+0000","last_fullsized":"2024-04-25T20:55:10.400580+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400744+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:55:10.400744+0000","last_peered":"2024-04-25T20:55:10.400744+0000","last_clean":"2024-04-25T20:55:10.400744+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:55:10.400744+0000","last_undegraded":"2024-04-25T20:55:10.400744+0000","last_fullsized":"2024-04-25T20:55:10.400744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.334414+0000","last_change":"2024-04-25T20:55:10.334414+0000","last_active":"2024-04-25T20:55:10.334414+0000","last_peered":"2024-04-25T20:55:10.334414+0000","last_clean":"2024-04-25T20:55:10.334414+0000","last_became_active":"2024-04-25T20:55:10.334022+0000","last_became_peered":"2024-04-25T20:55:10.334022+0000","last_unstale":"2024-04-25T20:55:10.334414+0000","last_undegraded":"2024-04-25T20:55:10.334414+0000","last_fullsized":"2024-04-25T20:55:10.334414+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:32.914876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325197+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:55:10.325197+0000","last_peered":"2024-04-25T20:55:10.325197+0000","last_clean":"2024-04-25T20:55:10.325197+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:55:10.325197+0000","last_undegraded":"2024-04-25T20:55:10.325197+0000","last_fullsized":"2024-04-25T20:55:10.325197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400426+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:55:10.400426+0000","last_peered":"2024-04-25T20:55:10.400426+0000","last_clean":"2024-04-25T20:55:10.400426+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:55:10.400426+0000","last_undegraded":"2024-04-25T20:55:10.400426+0000","last_fullsized":"2024-04-25T20:55:10.400426+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'28","reported_seq":115,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:12.394047+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:55:12.394047+0000","last_peered":"2024-04-25T20:55:12.394047+0000","last_clean":"2024-04-25T20:55:12.394047+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:55:12.394047+0000","last_undegraded":"2024-04-25T20:55:12.394047+0000","last_fullsized":"2024-04-25T20:55:12.394047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56090,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":62,"num_read_kb":85,"num_write":29,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332629+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:55:10.332629+0000","last_peered":"2024-04-25T20:55:10.332629+0000","last_clean":"2024-04-25T20:55:10.332629+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:55:10.332629+0000","last_undegraded":"2024-04-25T20:55:10.332629+0000","last_fullsized":"2024-04-25T20:55:10.332629+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331406+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:55:10.331406+0000","last_peered":"2024-04-25T20:55:10.331406+0000","last_clean":"2024-04-25T20:55:10.331406+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:55:10.331406+0000","last_undegraded":"2024-04-25T20:55:10.331406+0000","last_fullsized":"2024-04-25T20:55:10.331406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323434+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:55:10.323434+0000","last_peered":"2024-04-25T20:55:10.323434+0000","last_clean":"2024-04-25T20:55:10.323434+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:55:10.323434+0000","last_undegraded":"2024-04-25T20:55:10.323434+0000","last_fullsized":"2024-04-25T20:55:10.323434+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324988+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:55:10.324988+0000","last_peered":"2024-04-25T20:55:10.324988+0000","last_clean":"2024-04-25T20:55:10.324988+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:55:10.324988+0000","last_undegraded":"2024-04-25T20:55:10.324988+0000","last_fullsized":"2024-04-25T20:55:10.324988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400316+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:55:10.400316+0000","last_peered":"2024-04-25T20:55:10.400316+0000","last_clean":"2024-04-25T20:55:10.400316+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:55:10.400316+0000","last_undegraded":"2024-04-25T20:55:10.400316+0000","last_fullsized":"2024-04-25T20:55:10.400316+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":59,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:13.395702+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:55:13.395702+0000","last_peered":"2024-04-25T20:55:13.395702+0000","last_clean":"2024-04-25T20:55:13.395702+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:55:13.395702+0000","last_undegraded":"2024-04-25T20:55:13.395702+0000","last_fullsized":"2024-04-25T20:55:13.395702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400116+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:55:10.400116+0000","last_peered":"2024-04-25T20:55:10.400116+0000","last_clean":"2024-04-25T20:55:10.400116+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:55:10.400116+0000","last_undegraded":"2024-04-25T20:55:10.400116+0000","last_fullsized":"2024-04-25T20:55:10.400116+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332109+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:55:10.332109+0000","last_peered":"2024-04-25T20:55:10.332109+0000","last_clean":"2024-04-25T20:55:10.332109+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:55:10.332109+0000","last_undegraded":"2024-04-25T20:55:10.332109+0000","last_fullsized":"2024-04-25T20:55:10.332109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324690+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:55:10.324690+0000","last_peered":"2024-04-25T20:55:10.324690+0000","last_clean":"2024-04-25T20:55:10.324690+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:55:10.324690+0000","last_undegraded":"2024-04-25T20:55:10.324690+0000","last_fullsized":"2024-04-25T20:55:10.324690+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332534+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:55:10.332534+0000","last_peered":"2024-04-25T20:55:10.332534+0000","last_clean":"2024-04-25T20:55:10.332534+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:55:10.332534+0000","last_undegraded":"2024-04-25T20:55:10.332534+0000","last_fullsized":"2024-04-25T20:55:10.332534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.328107+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:55:10.328107+0000","last_peered":"2024-04-25T20:55:10.328107+0000","last_clean":"2024-04-25T20:55:10.328107+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:55:10.328107+0000","last_undegraded":"2024-04-25T20:55:10.328107+0000","last_fullsized":"2024-04-25T20:55:10.328107+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331313+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:55:10.331313+0000","last_peered":"2024-04-25T20:55:10.331313+0000","last_clean":"2024-04-25T20:55:10.331313+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:55:10.331313+0000","last_undegraded":"2024-04-25T20:55:10.331313+0000","last_fullsized":"2024-04-25T20:55:10.331313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326542+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:55:10.326542+0000","last_peered":"2024-04-25T20:55:10.326542+0000","last_clean":"2024-04-25T20:55:10.326542+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:55:10.326542+0000","last_undegraded":"2024-04-25T20:55:10.326542+0000","last_fullsized":"2024-04-25T20:55:10.326542+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400439+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:55:10.400439+0000","last_peered":"2024-04-25T20:55:10.400439+0000","last_clean":"2024-04-25T20:55:10.400439+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:55:10.400439+0000","last_undegraded":"2024-04-25T20:55:10.400439+0000","last_fullsized":"2024-04-25T20:55:10.400439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332432+0000","last_change":"2024-04-25T20:55:10.332432+0000","last_active":"2024-04-25T20:55:10.332432+0000","last_peered":"2024-04-25T20:55:10.332432+0000","last_clean":"2024-04-25T20:55:10.332432+0000","last_became_active":"2024-04-25T20:55:10.332033+0000","last_became_peered":"2024-04-25T20:55:10.332033+0000","last_unstale":"2024-04-25T20:55:10.332432+0000","last_undegraded":"2024-04-25T20:55:10.332432+0000","last_fullsized":"2024-04-25T20:55:10.332432+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:34.930705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325183+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:55:10.325183+0000","last_peered":"2024-04-25T20:55:10.325183+0000","last_clean":"2024-04-25T20:55:10.325183+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:55:10.325183+0000","last_undegraded":"2024-04-25T20:55:10.325183+0000","last_fullsized":"2024-04-25T20:55:10.325183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":56,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.390364+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:55:10.390364+0000","last_peered":"2024-04-25T20:55:10.390364+0000","last_clean":"2024-04-25T20:55:10.390364+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:55:10.390364+0000","last_undegraded":"2024-04-25T20:55:10.390364+0000","last_fullsized":"2024-04-25T20:55:10.390364+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330572+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:55:10.330572+0000","last_peered":"2024-04-25T20:55:10.330572+0000","last_clean":"2024-04-25T20:55:10.330572+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:55:10.330572+0000","last_undegraded":"2024-04-25T20:55:10.330572+0000","last_fullsized":"2024-04-25T20:55:10.330572+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324521+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:55:10.324521+0000","last_peered":"2024-04-25T20:55:10.324521+0000","last_clean":"2024-04-25T20:55:10.324521+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:55:10.324521+0000","last_undegraded":"2024-04-25T20:55:10.324521+0000","last_fullsized":"2024-04-25T20:55:10.324521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":361,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400226+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:55:10.400226+0000","last_peered":"2024-04-25T20:55:10.400226+0000","last_clean":"2024-04-25T20:55:10.400226+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:55:10.400226+0000","last_undegraded":"2024-04-25T20:55:10.400226+0000","last_fullsized":"2024-04-25T20:55:10.400226+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323353+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:55:10.323353+0000","last_peered":"2024-04-25T20:55:10.323353+0000","last_clean":"2024-04-25T20:55:10.323353+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:55:10.323353+0000","last_undegraded":"2024-04-25T20:55:10.323353+0000","last_fullsized":"2024-04-25T20:55:10.323353+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400305+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:55:10.400305+0000","last_peered":"2024-04-25T20:55:10.400305+0000","last_clean":"2024-04-25T20:55:10.400305+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:55:10.400305+0000","last_undegraded":"2024-04-25T20:55:10.400305+0000","last_fullsized":"2024-04-25T20:55:10.400305+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.337866+0000","last_change":"2024-04-25T20:55:10.337450+0000","last_active":"2024-04-25T20:55:10.337866+0000","last_peered":"2024-04-25T20:55:10.337866+0000","last_clean":"2024-04-25T20:55:10.337866+0000","last_became_active":"2024-04-25T20:55:10.332821+0000","last_became_peered":"2024-04-25T20:55:10.332821+0000","last_unstale":"2024-04-25T20:55:10.337866+0000","last_undegraded":"2024-04-25T20:55:10.337866+0000","last_fullsized":"2024-04-25T20:55:10.337866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:55.070702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323526+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:55:10.323526+0000","last_peered":"2024-04-25T20:55:10.323526+0000","last_clean":"2024-04-25T20:55:10.323526+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:55:10.323526+0000","last_undegraded":"2024-04-25T20:55:10.323526+0000","last_fullsized":"2024-04-25T20:55:10.323526+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"92'10","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331236+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:55:10.331236+0000","last_peered":"2024-04-25T20:55:10.331236+0000","last_clean":"2024-04-25T20:55:10.331236+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:55:10.331236+0000","last_undegraded":"2024-04-25T20:55:10.331236+0000","last_fullsized":"2024-04-25T20:55:10.331236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330886+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:55:10.330886+0000","last_peered":"2024-04-25T20:55:10.330886+0000","last_clean":"2024-04-25T20:55:10.330886+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:55:10.330886+0000","last_undegraded":"2024-04-25T20:55:10.330886+0000","last_fullsized":"2024-04-25T20:55:10.330886+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324679+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:55:10.324679+0000","last_peered":"2024-04-25T20:55:10.324679+0000","last_clean":"2024-04-25T20:55:10.324679+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:55:10.324679+0000","last_undegraded":"2024-04-25T20:55:10.324679+0000","last_fullsized":"2024-04-25T20:55:10.324679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326363+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:55:10.326363+0000","last_peered":"2024-04-25T20:55:10.326363+0000","last_clean":"2024-04-25T20:55:10.326363+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:55:10.326363+0000","last_undegraded":"2024-04-25T20:55:10.326363+0000","last_fullsized":"2024-04-25T20:55:10.326363+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.399951+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:55:10.399951+0000","last_peered":"2024-04-25T20:55:10.399951+0000","last_clean":"2024-04-25T20:55:10.399951+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:55:10.399951+0000","last_undegraded":"2024-04-25T20:55:10.399951+0000","last_fullsized":"2024-04-25T20:55:10.399951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331875+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:55:10.331875+0000","last_peered":"2024-04-25T20:55:10.331875+0000","last_clean":"2024-04-25T20:55:10.331875+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:55:10.331875+0000","last_undegraded":"2024-04-25T20:55:10.331875+0000","last_fullsized":"2024-04-25T20:55:10.331875+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330572+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:55:10.330572+0000","last_peered":"2024-04-25T20:55:10.330572+0000","last_clean":"2024-04-25T20:55:10.330572+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:55:10.330572+0000","last_undegraded":"2024-04-25T20:55:10.330572+0000","last_fullsized":"2024-04-25T20:55:10.330572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.327888+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:55:10.327888+0000","last_peered":"2024-04-25T20:55:10.327888+0000","last_clean":"2024-04-25T20:55:10.327888+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:55:10.327888+0000","last_undegraded":"2024-04-25T20:55:10.327888+0000","last_fullsized":"2024-04-25T20:55:10.327888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.340166+0000","last_change":"2024-04-25T20:55:10.339729+0000","last_active":"2024-04-25T20:55:10.340166+0000","last_peered":"2024-04-25T20:55:10.340166+0000","last_clean":"2024-04-25T20:55:10.340166+0000","last_became_active":"2024-04-25T20:55:10.333756+0000","last_became_peered":"2024-04-25T20:55:10.333756+0000","last_unstale":"2024-04-25T20:55:10.340166+0000","last_undegraded":"2024-04-25T20:55:10.340166+0000","last_fullsized":"2024-04-25T20:55:10.340166+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:48.806227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330388+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:55:10.330388+0000","last_peered":"2024-04-25T20:55:10.330388+0000","last_clean":"2024-04-25T20:55:10.330388+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:55:10.330388+0000","last_undegraded":"2024-04-25T20:55:10.330388+0000","last_fullsized":"2024-04-25T20:55:10.330388+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331929+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:55:10.331929+0000","last_peered":"2024-04-25T20:55:10.331929+0000","last_clean":"2024-04-25T20:55:10.331929+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:55:10.331929+0000","last_undegraded":"2024-04-25T20:55:10.331929+0000","last_fullsized":"2024-04-25T20:55:10.331929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400510+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:55:10.400510+0000","last_peered":"2024-04-25T20:55:10.400510+0000","last_clean":"2024-04-25T20:55:10.400510+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:55:10.400510+0000","last_undegraded":"2024-04-25T20:55:10.400510+0000","last_fullsized":"2024-04-25T20:55:10.400510+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331766+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:55:10.331766+0000","last_peered":"2024-04-25T20:55:10.331766+0000","last_clean":"2024-04-25T20:55:10.331766+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:55:10.331766+0000","last_undegraded":"2024-04-25T20:55:10.331766+0000","last_fullsized":"2024-04-25T20:55:10.331766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400938+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:55:10.400938+0000","last_peered":"2024-04-25T20:55:10.400938+0000","last_clean":"2024-04-25T20:55:10.400938+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:55:10.400938+0000","last_undegraded":"2024-04-25T20:55:10.400938+0000","last_fullsized":"2024-04-25T20:55:10.400938+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325055+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:55:10.325055+0000","last_peered":"2024-04-25T20:55:10.325055+0000","last_clean":"2024-04-25T20:55:10.325055+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:55:10.325055+0000","last_undegraded":"2024-04-25T20:55:10.325055+0000","last_fullsized":"2024-04-25T20:55:10.325055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'1","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400705+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:55:10.400705+0000","last_peered":"2024-04-25T20:55:10.400705+0000","last_clean":"2024-04-25T20:55:10.400705+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:55:10.400705+0000","last_undegraded":"2024-04-25T20:55:10.400705+0000","last_fullsized":"2024-04-25T20:55:10.400705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323575+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:55:10.323575+0000","last_peered":"2024-04-25T20:55:10.323575+0000","last_clean":"2024-04-25T20:55:10.323575+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:55:10.323575+0000","last_undegraded":"2024-04-25T20:55:10.323575+0000","last_fullsized":"2024-04-25T20:55:10.323575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400910+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:55:10.400910+0000","last_peered":"2024-04-25T20:55:10.400910+0000","last_clean":"2024-04-25T20:55:10.400910+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:55:10.400910+0000","last_undegraded":"2024-04-25T20:55:10.400910+0000","last_fullsized":"2024-04-25T20:55:10.400910+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.327935+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:55:10.327935+0000","last_peered":"2024-04-25T20:55:10.327935+0000","last_clean":"2024-04-25T20:55:10.327935+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:55:10.327935+0000","last_undegraded":"2024-04-25T20:55:10.327935+0000","last_fullsized":"2024-04-25T20:55:10.327935+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331088+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:55:10.331088+0000","last_peered":"2024-04-25T20:55:10.331088+0000","last_clean":"2024-04-25T20:55:10.331088+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:55:10.331088+0000","last_undegraded":"2024-04-25T20:55:10.331088+0000","last_fullsized":"2024-04-25T20:55:10.331088+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330927+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:55:10.330927+0000","last_peered":"2024-04-25T20:55:10.330927+0000","last_clean":"2024-04-25T20:55:10.330927+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:55:10.330927+0000","last_undegraded":"2024-04-25T20:55:10.330927+0000","last_fullsized":"2024-04-25T20:55:10.330927+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326436+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:55:10.326436+0000","last_peered":"2024-04-25T20:55:10.326436+0000","last_clean":"2024-04-25T20:55:10.326436+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:55:10.326436+0000","last_undegraded":"2024-04-25T20:55:10.326436+0000","last_fullsized":"2024-04-25T20:55:10.326436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332688+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:55:10.332688+0000","last_peered":"2024-04-25T20:55:10.332688+0000","last_clean":"2024-04-25T20:55:10.332688+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:55:10.332688+0000","last_undegraded":"2024-04-25T20:55:10.332688+0000","last_fullsized":"2024-04-25T20:55:10.332688+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.328326+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:55:10.328326+0000","last_peered":"2024-04-25T20:55:10.328326+0000","last_clean":"2024-04-25T20:55:10.328326+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:55:10.328326+0000","last_undegraded":"2024-04-25T20:55:10.328326+0000","last_fullsized":"2024-04-25T20:55:10.328326+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331638+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:55:10.331638+0000","last_peered":"2024-04-25T20:55:10.331638+0000","last_clean":"2024-04-25T20:55:10.331638+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:55:10.331638+0000","last_undegraded":"2024-04-25T20:55:10.331638+0000","last_fullsized":"2024-04-25T20:55:10.331638+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330830+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:55:10.330830+0000","last_peered":"2024-04-25T20:55:10.330830+0000","last_clean":"2024-04-25T20:55:10.330830+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:55:10.330830+0000","last_undegraded":"2024-04-25T20:55:10.330830+0000","last_fullsized":"2024-04-25T20:55:10.330830+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324727+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:55:10.324727+0000","last_peered":"2024-04-25T20:55:10.324727+0000","last_clean":"2024-04-25T20:55:10.324727+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:55:10.324727+0000","last_undegraded":"2024-04-25T20:55:10.324727+0000","last_fullsized":"2024-04-25T20:55:10.324727+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330451+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:55:10.330451+0000","last_peered":"2024-04-25T20:55:10.330451+0000","last_clean":"2024-04-25T20:55:10.330451+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:55:10.330451+0000","last_undegraded":"2024-04-25T20:55:10.330451+0000","last_fullsized":"2024-04-25T20:55:10.330451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400164+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:55:10.400164+0000","last_peered":"2024-04-25T20:55:10.400164+0000","last_clean":"2024-04-25T20:55:10.400164+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:55:10.400164+0000","last_undegraded":"2024-04-25T20:55:10.400164+0000","last_fullsized":"2024-04-25T20:55:10.400164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323589+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:55:10.323589+0000","last_peered":"2024-04-25T20:55:10.323589+0000","last_clean":"2024-04-25T20:55:10.323589+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:55:10.323589+0000","last_undegraded":"2024-04-25T20:55:10.323589+0000","last_fullsized":"2024-04-25T20:55:10.323589+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.328013+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:55:10.328013+0000","last_peered":"2024-04-25T20:55:10.328013+0000","last_clean":"2024-04-25T20:55:10.328013+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:55:10.328013+0000","last_undegraded":"2024-04-25T20:55:10.328013+0000","last_fullsized":"2024-04-25T20:55:10.328013+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60091,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":187,"num_read_kb":156,"num_write":161,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":405504,"data_stored":180273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285594,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1015808,"data_stored":495217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547231,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32848,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712304,"statfs":{"total":95995035648,"available":95961399296,"internally_reserved":0,"allocated":1703936,"data_stored":1142766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743459,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36344,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708808,"statfs":{"total":95995035648,"available":95957819392,"internally_reserved":0,"allocated":1028096,"data_stored":495431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005095,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":495329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1572,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1609728,"data_stored":1078896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36336,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708816,"statfs":{"total":95995035648,"available":95957827584,"internally_reserved":0,"allocated":1015808,"data_stored":494820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1073152,"data_stored":550703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":496091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1097728,"data_stored":552444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":495131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":495257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1634304,"data_stored":1086965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":57831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":57668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:55:17.668 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T20:55:17.669 INFO:tasks.cephadm:Setting up client nodes... 2024-04-25T20:55:17.669 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-25T20:55:17.906 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:18 smithi097 ceph-mon[32667]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:55:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:18 smithi097 ceph-mon[32667]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:18 smithi139 ceph-mon[33027]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:55:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:18 smithi139 ceph-mon[33027]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:18 smithi096 ceph-mon[29288]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:55:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:18 smithi096 ceph-mon[29288]: pgmap v327: 97 pgs: 97 active+clean; 635 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1023 B/s wr, 5 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:19.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 -- 172.21.15.96:0/860332933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 msgr2=0x7f99341041c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:19.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 --2- 172.21.15.96:0/860332933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341041c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9924009990 tx=0x7f9924009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 -- 172.21.15.96:0/860332933 shutdown_connections 2024-04-25T20:55:19.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 --2- 172.21.15.96:0/860332933 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 0x7f993410afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.602 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 --2- 172.21.15.96:0/860332933 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9934100d80 0x7f99341011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 --2- 172.21.15.96:0/860332933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341041c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 -- 172.21.15.96:0/860332933 >> 172.21.15.96:0/860332933 conn(0x7f99340fc6e0 msgr2=0x7f99340feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 -- 172.21.15.96:0/860332933 shutdown_connections 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 -- 172.21.15.96:0/860332933 wait complete. 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.599+0000 7f993bb34700 1 Processor -- start 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 -- start start 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9934100d80 0x7f9934198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 0x7f99341a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f993410e5c0 con 0x7f9934103d60 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f993410e440 con 0x7f9934104700 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f993410e740 con 0x7f9934100d80 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:19.603 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33770/0 (socket says 172.21.15.96:33770) 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 -- 172.21.15.96:0/3651544907 learned_addr learned my addr 172.21.15.96:0/3651544907 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993a0d1700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 0x7f99341a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 -- 172.21.15.96:0/3651544907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9934100d80 msgr2=0x7f9934198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99398d0700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9934100d80 0x7f9934198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9934100d80 0x7f9934198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 -- 172.21.15.96:0/3651544907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 msgr2=0x7f99341a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 0x7f99341a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 -- 172.21.15.96:0/3651544907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99341a0800 con 0x7f9934103d60 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993a0d1700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 0x7f99341a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f99390cf700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341990e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f99240038a0 tx=0x7f99240039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9924027070 con 0x7f9934103d60 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9924022470 con 0x7f9934103d60 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992400f6c0 con 0x7f9934103d60 2024-04-25T20:55:19.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.600+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99341a0a90 con 0x7f9934103d60 2024-04-25T20:55:19.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.601+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99341a0f10 con 0x7f9934103d60 2024-04-25T20:55:19.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.601+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9934050f80 con 0x7f9934103d60 2024-04-25T20:55:19.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.603+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f992402a070 con 0x7f9934103d60 2024-04-25T20:55:19.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.604+0000 7f992affd700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f99200a3d80 0x7f99200a6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:19.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.604+0000 7f99398d0700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f99200a3d80 0x7f99200a6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:19.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.604+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f99240d1860 con 0x7f9934103d60 2024-04-25T20:55:19.606 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.604+0000 7f99398d0700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f99200a3d80 0x7f99200a6240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f993406a520 tx=0x7f99300072f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:19.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.608+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9924013070 con 0x7f9934103d60 2024-04-25T20:55:19.890 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.889+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f993402e0a0 con 0x7f9934103d60 2024-04-25T20:55:19.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.892+0000 7f992affd700 1 -- 172.21.15.96:0/3651544907 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f993402e0a0 con 0x7f9934103d60 2024-04-25T20:55:19.894 INFO:teuthology.orchestra.run.smithi096.stdout:[client.0] 2024-04-25T20:55:19.894 INFO:teuthology.orchestra.run.smithi096.stdout: key = AQA3wypmMKIdNRAA6ViV1qg3RFQTW2vg/UsSFw== 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f99200a3d80 msgr2=0x7f99200a6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f99200a3d80 0x7f99200a6240 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f993406a520 tx=0x7f99300072f0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 msgr2=0x7f99341990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341990e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f99240038a0 tx=0x7f99240039f0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 shutdown_connections 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f99200a3d80 0x7f99200a6240 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9934104700 0x7f99341a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9934100d80 0x7f9934198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 --2- 172.21.15.96:0/3651544907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9934103d60 0x7f99341990e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:19.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 >> 172.21.15.96:0/3651544907 conn(0x7f99340fc6e0 msgr2=0x7f9934107060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:19.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 shutdown_connections 2024-04-25T20:55:19.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:19.894+0000 7f993bb34700 1 -- 172.21.15.96:0/3651544907 wait complete. 2024-04-25T20:55:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:20 smithi097 ceph-mon[32667]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:20 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3651544907' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T20:55:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:20 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3651544907' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T20:55:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:20 smithi139 ceph-mon[33027]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:20.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:20 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3651544907' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T20:55:20.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:20 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3651544907' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T20:55:20.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:20 smithi096 ceph-mon[29288]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.3 KiB/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:20.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:20 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3651544907' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-25T20:55:20.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:20 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3651544907' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-25T20:55:20.665 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:55:20.665 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-25T20:55:20.665 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T20:55:20.700 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T20:55:20.700 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-25T20:55:20.701 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-25T20:55:20.999 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:21.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:21 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:21.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:21 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:21.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:21 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:55:22.449 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:22 smithi096 ceph-mon[29288]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:22.449 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.449 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.449 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.449 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:22 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:22 smithi097 ceph-mon[32667]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:22 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:22 smithi139 ceph-mon[33027]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 2 B/s, 0 objects/s recovering 2024-04-25T20:55:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:22 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 -- 172.21.15.96:0/1685857413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06bba0 msgr2=0x7f59fc10f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 --2- 172.21.15.96:0/1685857413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06bba0 0x7f59fc10f900 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f59f400b0a0 tx=0x7f59f400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 -- 172.21.15.96:0/1685857413 shutdown_connections 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 --2- 172.21.15.96:0/1685857413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06bba0 0x7f59fc10f900 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 --2- 172.21.15.96:0/1685857413 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc108a30 0x7f59fc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 --2- 172.21.15.96:0/1685857413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59fc06b200 0x7f59fc06b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 -- 172.21.15.96:0/1685857413 >> 172.21.15.96:0/1685857413 conn(0x7f59fc0788f0 msgr2=0x7f59fc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 -- 172.21.15.96:0/1685857413 shutdown_connections 2024-04-25T20:55:22.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.864+0000 7f5a012d0700 1 -- 172.21.15.96:0/1685857413 wait complete. 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 Processor -- start 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 -- start start 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 0x7f59fc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc06bba0 0x7f59fc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59fc108a30 0x7f59fc19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112c70 con 0x7f59fc06b200 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112af0 con 0x7f59fc108a30 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f5a012d0700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc112df0 con 0x7f59fc06bba0 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f59faffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 0x7f59fc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f59faffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 0x7f59fc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33792/0 (socket says 172.21.15.96:33792) 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f59faffd700 1 -- 172.21.15.96:0/887107577 learned_addr learned my addr 172.21.15.96:0/887107577 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.865+0000 7f59fa7fc700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc06bba0 0x7f59fc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59faffd700 1 -- 172.21.15.96:0/887107577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc06bba0 msgr2=0x7f59fc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59faffd700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc06bba0 0x7f59fc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:22.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59faffd700 1 -- 172.21.15.96:0/887107577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59fc108a30 msgr2=0x7f59fc19d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59faffd700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59fc108a30 0x7f59fc19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59faffd700 1 -- 172.21.15.96:0/887107577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc19e110 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59faffd700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 0x7f59fc199100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00d950 tx=0x7f59ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec016070 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc19e400 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ec009df0 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec00f3c0 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.866+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a5c80 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.867+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc069730 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.869+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f59ec019070 con 0x7f59fc06b200 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.870+0000 7f59e3fff700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f59e40a3cd0 0x7f59e40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.870+0000 7f59fa7fc700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f59e40a3cd0 0x7f59e40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:22.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.870+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f59ec0d0060 con 0x7f59fc06b200 2024-04-25T20:55:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.870+0000 7f59fa7fc700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f59e40a3cd0 0x7f59e40a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59fc078e60 tx=0x7f59f0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:22.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:22.873+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59ec09c9e0 con 0x7f59fc06b200 2024-04-25T20:55:23.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.133+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f59fc19a460 con 0x7f59fc06b200 2024-04-25T20:55:23.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.138+0000 7f59e3fff700 1 -- 172.21.15.96:0/887107577 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264991 (secure 0 0 0) 0x7f59ec09c2e0 con 0x7f59fc06b200 2024-04-25T20:55:23.142 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f59e40a3cd0 msgr2=0x7f59e40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f59e40a3cd0 0x7f59e40a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f59fc078e60 tx=0x7f59f0009190 comp rx=0 tx=0).stop 2024-04-25T20:55:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 msgr2=0x7f59fc199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 0x7f59fc199100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00d950 tx=0x7f59ec00dc60 comp rx=0 tx=0).stop 2024-04-25T20:55:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 shutdown_connections 2024-04-25T20:55:23.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f59e40a3cd0 0x7f59e40a6190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59fc108a30 0x7f59fc19d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59fc06b200 0x7f59fc199100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 --2- 172.21.15.96:0/887107577 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f59fc06bba0 0x7f59fc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 >> 172.21.15.96:0/887107577 conn(0x7f59fc0788f0 msgr2=0x7f59fc0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 shutdown_connections 2024-04-25T20:55:23.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:23.141+0000 7f5a012d0700 1 -- 172.21.15.96:0/887107577 wait complete. 2024-04-25T20:55:23.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:23 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/887107577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T20:55:23.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:23 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/887107577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T20:55:23.628 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:23 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/887107577' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-25T20:55:23.898 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":23858312},{"type":"v1","addr":"172.21.15.96:6801","nonce":23858312}]},"active_addr":"172.21.15.96:6801/23858312","active_change":"2024-04-25T20:47:09.141171+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.96:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2934738303}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":2385969372}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1528469114}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":995634760}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.96:0","nonce":1944404771}]}]} 2024-04-25T20:55:23.904 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-25T20:55:23.904 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-25T20:55:23.905 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:55:24.168 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:24 smithi097 ceph-mon[32667]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T20:55:24.614 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:24 smithi096 ceph-mon[29288]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T20:55:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:24 smithi139 ceph-mon[33027]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 -- 172.21.15.96:0/1580745562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334102a90 msgr2=0x7f2334102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 --2- 172.21.15.96:0/1580745562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334102a90 0x7f2334102f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2328009990 tx=0x7f2328009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 -- 172.21.15.96:0/1580745562 shutdown_connections 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 --2- 172.21.15.96:0/1580745562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334103450 0x7f233410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 --2- 172.21.15.96:0/1580745562 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2334108a50 0x7f2334108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 --2- 172.21.15.96:0/1580745562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334102a90 0x7f2334102f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 -- 172.21.15.96:0/1580745562 >> 172.21.15.96:0/1580745562 conn(0x7f2334079ba0 msgr2=0x7f2334079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 -- 172.21.15.96:0/1580745562 shutdown_connections 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.890+0000 7f2339382700 1 -- 172.21.15.96:0/1580745562 wait complete. 2024-04-25T20:55:25.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.891+0000 7f2339382700 1 Processor -- start 2024-04-25T20:55:25.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.891+0000 7f2339382700 1 -- start start 2024-04-25T20:55:25.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2339382700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2334102a90 0x7f2334199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2339382700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 0x7f23341997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2339382700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334108a50 0x7f233419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2339382700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2334112ce0 con 0x7f2334103450 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2339382700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2334112b60 con 0x7f2334108a50 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2339382700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2334112e60 con 0x7f2334102a90 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 0x7f23341997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 0x7f23341997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33810/0 (socket says 172.21.15.96:33810) 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 -- 172.21.15.96:0/894562038 learned_addr learned my addr 172.21.15.96:0/894562038 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23337fe700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334108a50 0x7f233419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f2332ffd700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2334102a90 0x7f2334199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 -- 172.21.15.96:0/894562038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2334102a90 msgr2=0x7f2334199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2334102a90 0x7f2334199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 -- 172.21.15.96:0/894562038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334108a50 msgr2=0x7f233419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:25.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334108a50 0x7f233419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 -- 172.21.15.96:0/894562038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f233419e280 con 0x7f2334103450 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23337fe700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334108a50 0x7f233419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f23327fc700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 0x7f23341997b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f23280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.892+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2328027070 con 0x7f2334103450 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.893+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f233419e510 con 0x7f2334103450 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.893+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23341a5df0 con 0x7f2334103450 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.893+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23280041d0 con 0x7f2334103450 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.893+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f232800f460 con 0x7f2334103450 2024-04-25T20:55:25.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.894+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2320005330 con 0x7f2334103450 2024-04-25T20:55:25.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.895+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f232802a070 con 0x7f2334103450 2024-04-25T20:55:25.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.896+0000 7f231bfff700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f231c0a3d50 0x7f231c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:25.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.896+0000 7f2332ffd700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f231c0a3d50 0x7f231c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:25.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.896+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f23280d17b0 con 0x7f2334103450 2024-04-25T20:55:25.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.896+0000 7f2332ffd700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f231c0a3d50 0x7f231c0a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2324006fd0 tx=0x7f2324008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:25.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:25.899+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2328031070 con 0x7f2334103450 2024-04-25T20:55:26.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.105+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f23200058f0 con 0x7f2334103450 2024-04-25T20:55:26.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.106+0000 7f231bfff700 1 -- 172.21.15.96:0/894562038 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22523 (secure 0 0 0) 0x7f2328013070 con 0x7f2334103450 2024-04-25T20:55:26.107 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:26.108 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":93,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:55:10.320396+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:55:26.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f231c0a3d50 msgr2=0x7f231c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:26.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f231c0a3d50 0x7f231c0a6210 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2324006fd0 tx=0x7f2324008040 comp rx=0 tx=0).stop 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 msgr2=0x7f23341997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 0x7f23341997b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2328000c00 tx=0x7f23280047b0 comp rx=0 tx=0).stop 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 shutdown_connections 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f231c0a3d50 0x7f231c0a6210 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2334108a50 0x7f233419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2334102a90 0x7f2334199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 --2- 172.21.15.96:0/894562038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2334103450 0x7f23341997b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.108+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 >> 172.21.15.96:0/894562038 conn(0x7f2334079ba0 msgr2=0x7f2334101960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.109+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 shutdown_connections 2024-04-25T20:55:26.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:26.109+0000 7f2339382700 1 -- 172.21.15.96:0/894562038 wait complete. 2024-04-25T20:55:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:26 smithi139 ceph-mon[33027]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T20:55:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:26 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:26.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:26 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/894562038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:26 smithi096 ceph-mon[29288]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T20:55:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:26 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/894562038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:26.810 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-25T20:55:26.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:55:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:26 smithi097 ceph-mon[32667]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 1 B/s, 0 objects/s recovering 2024-04-25T20:55:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:26 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:26.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:26 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/894562038' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:27.046 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:27.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:27 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:27.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:27 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:27.420 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:27 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:27 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:27 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:27 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:27 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:55:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:27 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:55:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:27 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:55:28.464 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:28 smithi096 ceph-mon[29288]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T20:55:28.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:28 smithi139 ceph-mon[33027]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 -- 172.21.15.96:0/3827739235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 msgr2=0x7fd6c410f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3827739235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c410f8b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00b0a0 tx=0x7fd6bc00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 -- 172.21.15.96:0/3827739235 shutdown_connections 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3827739235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c410f8b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3827739235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4108a50 0x7fd6c4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3827739235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c4102a90 0x7fd6c4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 -- 172.21.15.96:0/3827739235 >> 172.21.15.96:0/3827739235 conn(0x7fd6c4079ba0 msgr2=0x7fd6c4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 -- 172.21.15.96:0/3827739235 shutdown_connections 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 -- 172.21.15.96:0/3827739235 wait complete. 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 Processor -- start 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.693+0000 7fd6ca871700 1 -- start start 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4102a90 0x7fd6c4199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c4199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c4108a50 0x7fd6c419da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c4112c80 con 0x7fd6c4103450 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c4112b00 con 0x7fd6c4102a90 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c4112e00 con 0x7fd6c4108a50 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c4199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c4199690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37786/0 (socket says 172.21.15.96:37786) 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 -- 172.21.15.96:0/3975120775 learned_addr learned my addr 172.21.15.96:0/3975120775 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:28.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c8e0e700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c4108a50 0x7fd6c419da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c3fff700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4102a90 0x7fd6c4199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 -- 172.21.15.96:0/3975120775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c4108a50 msgr2=0x7fd6c419da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c4108a50 0x7fd6c419da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 -- 172.21.15.96:0/3975120775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4102a90 msgr2=0x7fd6c4199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4102a90 0x7fd6c4199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 -- 172.21.15.96:0/3975120775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c419e160 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c3fff700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4102a90 0x7fd6c4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c37fe700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c4199690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800d980 tx=0x7fd6b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8017070 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6b8005020 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.694+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c419e450 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.695+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8009b50 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.695+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c41a5cd0 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.696+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c4069730 con 0x7fd6c4103450 2024-04-25T20:55:28.712 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.697+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd6b80051d0 con 0x7fd6c4103450 2024-04-25T20:55:28.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.697+0000 7fd6c17fa700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd6ac0a3cd0 0x7fd6ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:28.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.697+0000 7fd6c3fff700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd6ac0a3cd0 0x7fd6ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:28.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.697+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fd6b80d03d0 con 0x7fd6c4103450 2024-04-25T20:55:28.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.698+0000 7fd6c3fff700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd6ac0a3cd0 0x7fd6ac0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4007900 tx=0x7fd6b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:28.713 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.702+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6b8020020 con 0x7fd6c4103450 2024-04-25T20:55:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:28 smithi097 ceph-mon[32667]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-04-25T20:55:28.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.902+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd6c402df80 con 0x7fd6c4103450 2024-04-25T20:55:28.904 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.904+0000 7fd6c17fa700 1 -- 172.21.15.96:0/3975120775 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22523 (secure 0 0 0) 0x7fd6b809cd50 con 0x7fd6c4103450 2024-04-25T20:55:28.905 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:55:28.905 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":93,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:55:10.320396+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd6ac0a3cd0 msgr2=0x7fd6ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd6ac0a3cd0 0x7fd6ac0a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6b4007900 tx=0x7fd6b4008040 comp rx=0 tx=0).stop 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 msgr2=0x7fd6c4199690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c4199690 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800d980 tx=0x7fd6b800dc90 comp rx=0 tx=0).stop 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 shutdown_connections 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd6ac0a3cd0 0x7fd6ac0a6190 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd6c4108a50 0x7fd6c419da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6c4102a90 0x7fd6c4199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 --2- 172.21.15.96:0/3975120775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd6c4103450 0x7fd6c4199690 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 >> 172.21.15.96:0/3975120775 conn(0x7fd6c4079ba0 msgr2=0x7fd6c4101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:28.907 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 shutdown_connections 2024-04-25T20:55:28.908 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:28.906+0000 7fd6ca871700 1 -- 172.21.15.96:0/3975120775 wait complete. 2024-04-25T20:55:29.664 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:29 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3975120775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:29.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-25T20:55:29.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-25T20:55:29.694 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-25T20:55:29.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-25T20:55:29.695 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-25T20:55:29.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-25T20:55:29.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-25T20:55:29.696 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-25T20:55:29.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-25T20:55:29.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-25T20:55:29.697 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-25T20:55:29.698 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-25T20:55:29.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:29 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3975120775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:29.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:29 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3975120775' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:55:30.435 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:30.442 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:30.664 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:30 smithi096 ceph-mon[29288]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:55:30.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:30 smithi097 ceph-mon[32667]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:55:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:30 smithi139 ceph-mon[33027]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:55:31.479 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.480 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.481 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.483 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.484 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.486 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:31.487 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 -- 172.21.15.96:0/2332881854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101dc0 msgr2=0x7fd34410f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 --2- 172.21.15.96:0/2332881854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101dc0 0x7fd34410f830 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd33c00b0a0 tx=0x7fd33c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 -- 172.21.15.96:0/2332881854 shutdown_connections 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 --2- 172.21.15.96:0/2332881854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101dc0 0x7fd34410f830 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 --2- 172.21.15.96:0/2332881854 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd344107b20 0x7fd344107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 --2- 172.21.15.96:0/2332881854 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344101400 0x7fd344101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 -- 172.21.15.96:0/2332881854 >> 172.21.15.96:0/2332881854 conn(0x7fd344076c60 msgr2=0x7fd344077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 -- 172.21.15.96:0/2332881854 shutdown_connections 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 -- 172.21.15.96:0/2332881854 wait complete. 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.394+0000 7fd349c75700 1 Processor -- start 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 -- start start 2024-04-25T20:55:32.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101400 0x7fd344198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd344101dc0 0x7fd344199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344107b20 0x7fd34419d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112c50 con 0x7fd344101400 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112ad0 con 0x7fd344107b20 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd349c75700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112dd0 con 0x7fd344101dc0 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101400 0x7fd344198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101400 0x7fd344198f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37806/0 (socket says 172.21.15.96:37806) 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 -- 172.21.15.96:0/457201769 learned_addr learned my addr 172.21.15.96:0/457201769 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd343fff700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344107b20 0x7fd34419d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 -- 172.21.15.96:0/457201769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd344101dc0 msgr2=0x7fd344199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd344101dc0 0x7fd344199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 -- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344107b20 msgr2=0x7fd34419d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd344107b20 0x7fd34419d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 -- 172.21.15.96:0/457201769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd34419df30 con 0x7fd344101400 2024-04-25T20:55:32.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd3437fe700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101400 0x7fd344198f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c990 tx=0x7fd33400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.395+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334007b00 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.396+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd334007ca0 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.396+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd34419e220 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.396+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334018660 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.396+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3441a5aa0 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.396+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd344100690 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.398+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd33400f450 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.398+0000 7fd340ff9700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd32c0a3cd0 0x7fd32c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.398+0000 7fd342ffd700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd32c0a3cd0 0x7fd32c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.398+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fd3340d0ab0 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.399+0000 7fd340ff9700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] conn(0x7fd32c0ad710 0x7fd32c0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.399+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 --> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd33401ee70 con 0x7fd32c0ad710 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.399+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fd3340d0f90 con 0x7fd344101400 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.399+0000 7fd342ffd700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd32c0a3cd0 0x7fd32c0a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd338005fd0 tx=0x7fd338005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.399+0000 7fd343fff700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] conn(0x7fd32c0ad710 0x7fd32c0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:32.404 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.399+0000 7fd343fff700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] conn(0x7fd32c0ad710 0x7fd32c0afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:32.405 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.400+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== osd.1 v2:172.21.15.97:6800/3234593794 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd33401ee70 con 0x7fd32c0ad710 2024-04-25T20:55:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.416+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 --> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd34406ac30 con 0x7fd32c0ad710 2024-04-25T20:55:32.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd340ff9700 1 -- 172.21.15.96:0/457201769 <== osd.1 v2:172.21.15.97:6800/3234593794 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd34406ac30 con 0x7fd32c0ad710 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] conn(0x7fd32c0ad710 msgr2=0x7fd32c0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] conn(0x7fd32c0ad710 0x7fd32c0afbb0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd32c0a3cd0 msgr2=0x7fd32c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd32c0a3cd0 0x7fd32c0a6190 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd338005fd0 tx=0x7fd338005d00 comp rx=0 tx=0).stop 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101400 msgr2=0x7fd344198f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 --2- 172.21.15.96:0/457201769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd344101400 0x7fd344198f20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd33400c990 tx=0x7fd33400cd50 comp rx=0 tx=0).stop 2024-04-25T20:55:32.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd343fff700 1 -- 172.21.15.96:0/457201769 reap_dead start 2024-04-25T20:55:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 shutdown_connections 2024-04-25T20:55:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 >> 172.21.15.96:0/457201769 conn(0x7fd344076c60 msgr2=0x7fd3440ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 shutdown_connections 2024-04-25T20:55:32.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:32.417+0000 7fd349c75700 1 -- 172.21.15.96:0/457201769 wait complete. 2024-04-25T20:55:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:32 smithi096 ceph-mon[29288]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:32.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:32 smithi097 ceph-mon[32667]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:32 smithi139 ceph-mon[33027]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:33.930 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:33.931 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:34.643 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:34 smithi096 ceph-mon[29288]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:34.670 INFO:teuthology.orchestra.run.smithi096.stdout:55834574928 2024-04-25T20:55:34.670 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-25T20:55:34.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:34 smithi097 ceph-mon[32667]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:34.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:34 smithi139 ceph-mon[33027]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:36 smithi096 ceph-mon[29288]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:36.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:36 smithi097 ceph-mon[32667]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:36.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:36 smithi139 ceph-mon[33027]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:38 smithi096 ceph-mon[29288]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:38.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:38 smithi097 ceph-mon[32667]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:38.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:38 smithi139 ceph-mon[33027]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:40.619 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:40 smithi096 ceph-mon[29288]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:55:40.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:40 smithi097 ceph-mon[32667]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:55:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:40 smithi139 ceph-mon[33027]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.474+0000 7f5ae5539700 1 -- 172.21.15.96:0/320160874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0107990 msgr2=0x7f5ae0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.474+0000 7f5ae5539700 1 --2- 172.21.15.96:0/320160874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0107990 0x7f5ae0107d70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009990 tx=0x7f5ad0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 -- 172.21.15.96:0/320160874 shutdown_connections 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 --2- 172.21.15.96:0/320160874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae0102300 0x7f5ae010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 --2- 172.21.15.96:0/320160874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0107990 0x7f5ae0107d70 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 --2- 172.21.15.96:0/320160874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ae0101940 0x7f5ae0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 -- 172.21.15.96:0/320160874 >> 172.21.15.96:0/320160874 conn(0x7f5ae00788f0 msgr2=0x7f5ae0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 -- 172.21.15.96:0/320160874 shutdown_connections 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 -- 172.21.15.96:0/320160874 wait complete. 2024-04-25T20:55:41.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 Processor -- start 2024-04-25T20:55:41.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.475+0000 7f5ae5539700 1 -- start start 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0101940 0x7f5ae01034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae0102300 0x7f5ae0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ae0107990 0x7f5ae0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0111bd0 con 0x7f5ae0101940 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0111a50 con 0x7f5ae0102300 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae0111d50 con 0x7f5ae0107990 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0101940 0x7f5ae01034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0101940 0x7f5ae01034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50988/0 (socket says 172.21.15.96:50988) 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 -- 172.21.15.96:0/1186625461 learned_addr learned my addr 172.21.15.96:0/1186625461 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adf7fe700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ae0107990 0x7f5ae0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ade7fc700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae0102300 0x7f5ae0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 -- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ae0107990 msgr2=0x7f5ae0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:41.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5ae0107990 0x7f5ae0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 -- 172.21.15.96:0/1186625461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae0102300 msgr2=0x7f5ae0103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae0102300 0x7f5ae0103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 -- 172.21.15.96:0/1186625461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae01a77e0 con 0x7f5ae0101940 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ade7fc700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ae0102300 0x7f5ae0103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5adeffd700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0101940 0x7f5ae01034f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0027070 con 0x7f5ae0101940 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae01a7a70 con 0x7f5ae0101940 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad00040e0 con 0x7f5ae0101940 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad000f460 con 0x7f5ae0101940 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.476+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae01a7f20 con 0x7f5ae0101940 2024-04-25T20:55:41.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.477+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5ae01068d0 con 0x7f5ae0101940 2024-04-25T20:55:41.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.478+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5ad002a070 con 0x7f5ae0101940 2024-04-25T20:55:41.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ac7fff700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ade7fc700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f5ad00d1b30 con 0x7f5ae0101940 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ac7fff700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] conn(0x7f5ac80ad710 0x7f5ac80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 --> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5ad0011ec0 con 0x7f5ac80ad710 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f5ad00d2010 con 0x7f5ae0101940 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.479+0000 7f5ade7fc700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0078f80 tx=0x7f5ad4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.480+0000 7f5adf7fe700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] conn(0x7f5ac80ad710 0x7f5ac80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.480+0000 7f5adf7fe700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] conn(0x7f5ac80ad710 0x7f5ac80afbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:41.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.481+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== osd.2 v2:172.21.15.139:6800/1811270506 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5ad0011ec0 con 0x7f5ac80ad710 2024-04-25T20:55:41.498 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.497+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 --> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5ae0103060 con 0x7f5ac80ad710 2024-04-25T20:55:41.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.498+0000 7f5ac7fff700 1 -- 172.21.15.96:0/1186625461 <== osd.2 v2:172.21.15.139:6800/1811270506 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5ae0103060 con 0x7f5ac80ad710 2024-04-25T20:55:41.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.498+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] conn(0x7f5ac80ad710 msgr2=0x7f5ac80afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:41.499 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] conn(0x7f5ac80ad710 0x7f5ac80afbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5ac80a3cd0 msgr2=0x7f5ac80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f5ac80a3cd0 0x7f5ac80a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0078f80 tx=0x7f5ad4008040 comp rx=0 tx=0).stop 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0101940 msgr2=0x7f5ae01034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 --2- 172.21.15.96:0/1186625461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ae0101940 0x7f5ae01034f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad00047b0 comp rx=0 tx=0).stop 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5adf7fe700 1 -- 172.21.15.96:0/1186625461 reap_dead start 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 shutdown_connections 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 >> 172.21.15.96:0/1186625461 conn(0x7f5ae00788f0 msgr2=0x7f5ae006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 shutdown_connections 2024-04-25T20:55:41.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:41.499+0000 7f5ae5539700 1 -- 172.21.15.96:0/1186625461 wait complete. 2024-04-25T20:55:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:42 smithi096 ceph-mon[29288]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:42 smithi097 ceph-mon[32667]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:42 smithi139 ceph-mon[33027]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:43.963 INFO:teuthology.orchestra.run.smithi096.stdout:77309411405 2024-04-25T20:55:43.964 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-25T20:55:44.701 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:44 smithi096 ceph-mon[29288]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:44.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 -- 172.21.15.96:0/2982176968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c106cc0 msgr2=0x7f374c10e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:44.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 --2- 172.21.15.96:0/2982176968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c106cc0 0x7f374c10e8d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f374400b0d0 tx=0x7f374400b3e0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 -- 172.21.15.96:0/2982176968 shutdown_connections 2024-04-25T20:55:44.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 --2- 172.21.15.96:0/2982176968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c106cc0 0x7f374c10e8d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 --2- 172.21.15.96:0/2982176968 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374c1023a0 0x7f374c102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 --2- 172.21.15.96:0/2982176968 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374c102d50 0x7f374c1065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 -- 172.21.15.96:0/2982176968 >> 172.21.15.96:0/2982176968 conn(0x7f374c077fb0 msgr2=0x7f374c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:44.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 -- 172.21.15.96:0/2982176968 shutdown_connections 2024-04-25T20:55:44.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.804+0000 7f375093c700 1 -- 172.21.15.96:0/2982176968 wait complete. 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 Processor -- start 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 -- start start 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374c1023a0 0x7f374c0fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c102d50 0x7f374c0fd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374c106cc0 0x7f374c0fdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111bc0 con 0x7f374c102d50 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111a40 con 0x7f374c1023a0 2024-04-25T20:55:44.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f375093c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111d40 con 0x7f374c106cc0 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f3749d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c102d50 0x7f374c0fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f3749d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c102d50 0x7f374c0fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51000/0 (socket says 172.21.15.96:51000) 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f3749d9b700 1 -- 172.21.15.96:0/509032339 learned_addr learned my addr 172.21.15.96:0/509032339 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f374ad9d700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374c106cc0 0x7f374c0fdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.805+0000 7f374a59c700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374c1023a0 0x7f374c0fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f3749d9b700 1 -- 172.21.15.96:0/509032339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374c106cc0 msgr2=0x7f374c0fdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f3749d9b700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f374c106cc0 0x7f374c0fdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f3749d9b700 1 -- 172.21.15.96:0/509032339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374c1023a0 msgr2=0x7f374c0fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f3749d9b700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f374c1023a0 0x7f374c0fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f3749d9b700 1 -- 172.21.15.96:0/509032339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c1a75f0 con 0x7f374c102d50 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f3749d9b700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c102d50 0x7f374c0fd960 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f374000c8f0 tx=0x7f374000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:44.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37400078d0 con 0x7f374c102d50 2024-04-25T20:55:44.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c1a78e0 con 0x7f374c102d50 2024-04-25T20:55:44.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f374000f450 con 0x7f374c102d50 2024-04-25T20:55:44.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740018660 con 0x7f374c102d50 2024-04-25T20:55:44.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.806+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c1a7e70 con 0x7f374c102d50 2024-04-25T20:55:44.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.807+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f374c063110 con 0x7f374c102d50 2024-04-25T20:55:44.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.808+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f374000f900 con 0x7f374c102d50 2024-04-25T20:55:44.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f373b7fe700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f37340a3d50 0x7f37340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:44.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f374a59c700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f37340a3d50 0x7f37340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f37400d03e0 con 0x7f374c102d50 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f373b7fe700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] conn(0x7f37340ad790 0x7f37340afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 --> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37340b0300 con 0x7f37340ad790 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f37400d08c0 con 0x7f374c102d50 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f374ad9d700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] conn(0x7f37340ad790 0x7f37340afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f374a59c700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f37340a3d50 0x7f37340a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f373c00ac50 tx=0x7f373c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.809+0000 7f374ad9d700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] conn(0x7f37340ad790 0x7f37340afc30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:44.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.810+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== osd.0 v2:172.21.15.96:6802/1267246354 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f37340b0300 con 0x7f37340ad790 2024-04-25T20:55:44.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 --> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f374c063590 con 0x7f37340ad790 2024-04-25T20:55:44.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f373b7fe700 1 -- 172.21.15.96:0/509032339 <== osd.0 v2:172.21.15.96:6802/1267246354 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f374c063590 con 0x7f37340ad790 2024-04-25T20:55:44.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] conn(0x7f37340ad790 msgr2=0x7f37340afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] conn(0x7f37340ad790 0x7f37340afc30 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f37340a3d50 msgr2=0x7f37340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f37340a3d50 0x7f37340a6210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f373c00ac50 tx=0x7f373c00a380 comp rx=0 tx=0).stop 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c102d50 msgr2=0x7f374c0fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 --2- 172.21.15.96:0/509032339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f374c102d50 0x7f374c0fd960 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f374000c8f0 tx=0x7f374000cc00 comp rx=0 tx=0).stop 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f374ad9d700 1 -- 172.21.15.96:0/509032339 reap_dead start 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 shutdown_connections 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 >> 172.21.15.96:0/509032339 conn(0x7f374c077fb0 msgr2=0x7f374c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 shutdown_connections 2024-04-25T20:55:44.828 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:44.827+0000 7f375093c700 1 -- 172.21.15.96:0/509032339 wait complete. 2024-04-25T20:55:44.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:44 smithi097 ceph-mon[32667]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:44.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:44 smithi139 ceph-mon[33027]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:46.565 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:46 smithi096 ceph-mon[29288]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:46.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:46 smithi097 ceph-mon[32667]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:46 smithi139 ceph-mon[33027]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.141+0000 7f9a206d6700 1 -- 172.21.15.96:0/59743960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 msgr2=0x7f9a1810f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- 172.21.15.96:0/59743960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1810f900 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000b0d0 tx=0x7f9a1000b3e0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- 172.21.15.96:0/59743960 shutdown_connections 2024-04-25T20:55:48.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- 172.21.15.96:0/59743960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1810f900 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- 172.21.15.96:0/59743960 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a18108a50 0x7f9a18108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- 172.21.15.96:0/59743960 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1806b1b0 0x7f9a1806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- 172.21.15.96:0/59743960 >> 172.21.15.96:0/59743960 conn(0x7f9a18077990 msgr2=0x7f9a18077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- 172.21.15.96:0/59743960 shutdown_connections 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- 172.21.15.96:0/59743960 wait complete. 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 Processor -- start 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- start start 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1806b1b0 0x7f9a1819b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1819b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a18108a50 0x7f9a1819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18112c30 con 0x7f9a1806bb70 2024-04-25T20:55:48.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18112ab0 con 0x7f9a1806b1b0 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.142+0000 7f9a206d6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9a18112db0 con 0x7f9a18108a50 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1819b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1819b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54842/0 (socket says 172.21.15.96:54842) 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 -- 172.21.15.96:0/1216619639 learned_addr learned my addr 172.21.15.96:0/1216619639 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1e472700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1806b1b0 0x7f9a1819b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1ec73700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a18108a50 0x7f9a1819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 -- 172.21.15.96:0/1216619639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a18108a50 msgr2=0x7f9a1819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9a18108a50 0x7f9a1819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 -- 172.21.15.96:0/1216619639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1806b1b0 msgr2=0x7f9a1819b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1806b1b0 0x7f9a1819b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 -- 172.21.15.96:0/1216619639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a181a0360 con 0x7f9a1806bb70 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1e472700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9a1806b1b0 0x7f9a1819b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:48.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a1dc71700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1819b890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800e9a0 tx=0x7f9a0800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:48.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0800cbd0 con 0x7f9a1806bb70 2024-04-25T20:55:48.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a181a0650 con 0x7f9a1806bb70 2024-04-25T20:55:48.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a0800cd70 con 0x7f9a1806bb70 2024-04-25T20:55:48.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a08018680 con 0x7f9a1806bb70 2024-04-25T20:55:48.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.143+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a181a7ed0 con 0x7f9a1806bb70 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.144+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a1810e5d0 con 0x7f9a1806bb70 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.146+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9a08010970 con 0x7f9a1806bb70 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.146+0000 7f9a0f7fe700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9a040a3cd0 0x7f9a040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.146+0000 7f9a1e472700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9a040a3cd0 0x7f9a040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.146+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f9a080d0fc0 con 0x7f9a1806bb70 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.147+0000 7f9a0f7fe700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] conn(0x7f9a040ad710 0x7f9a040afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.147+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 --> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9a040b0280 con 0x7f9a040ad710 2024-04-25T20:55:48.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.147+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f9a080d14a0 con 0x7f9a1806bb70 2024-04-25T20:55:48.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.147+0000 7f9a1ec73700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] conn(0x7f9a040ad710 0x7f9a040afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.147+0000 7f9a1e472700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9a040a3cd0 0x7f9a040a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005e10 tx=0x7f9a14005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:48.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.147+0000 7f9a1ec73700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] conn(0x7f9a040ad710 0x7f9a040afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:48.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.148+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== osd.9 v2:172.21.15.96:6826/2687652151 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9a040b0280 con 0x7f9a040ad710 2024-04-25T20:55:48.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.164+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 --> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a181041d0 con 0x7f9a040ad710 2024-04-25T20:55:48.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a0f7fe700 1 -- 172.21.15.96:0/1216619639 <== osd.9 v2:172.21.15.96:6826/2687652151 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a181041d0 con 0x7f9a040ad710 2024-04-25T20:55:48.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] conn(0x7f9a040ad710 msgr2=0x7f9a040afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] conn(0x7f9a040ad710 0x7f9a040afbb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9a040a3cd0 msgr2=0x7f9a040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9a040a3cd0 0x7f9a040a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9a14005e10 tx=0x7f9a14005d80 comp rx=0 tx=0).stop 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 msgr2=0x7f9a1819b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 --2- 172.21.15.96:0/1216619639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9a1806bb70 0x7f9a1819b890 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800e9a0 tx=0x7f9a0800ed60 comp rx=0 tx=0).stop 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a1ec73700 1 -- 172.21.15.96:0/1216619639 reap_dead start 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 shutdown_connections 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 >> 172.21.15.96:0/1216619639 conn(0x7f9a18077990 msgr2=0x7f9a1810dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 shutdown_connections 2024-04-25T20:55:48.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.165+0000 7f9a206d6700 1 -- 172.21.15.96:0/1216619639 wait complete. 2024-04-25T20:55:48.507 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:48 smithi096 ceph-mon[29288]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:48.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 -- 172.21.15.96:0/3859448239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880101950 msgr2=0x7f088010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 --2- 172.21.15.96:0/3859448239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880101950 0x7f088010f920 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f08700099c0 tx=0x7f0870009cd0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 -- 172.21.15.96:0/3859448239 shutdown_connections 2024-04-25T20:55:48.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 --2- 172.21.15.96:0/3859448239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880101950 0x7f088010f920 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.802 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 --2- 172.21.15.96:0/3859448239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0880107a60 0x7f0880107e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 --2- 172.21.15.96:0/3859448239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880100f90 0x7f0880101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 -- 172.21.15.96:0/3859448239 >> 172.21.15.96:0/3859448239 conn(0x7f0880077f40 msgr2=0x7f08800ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 -- 172.21.15.96:0/3859448239 shutdown_connections 2024-04-25T20:55:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 -- 172.21.15.96:0/3859448239 wait complete. 2024-04-25T20:55:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 Processor -- start 2024-04-25T20:55:48.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.801+0000 7f08881ca700 1 -- start start 2024-04-25T20:55:48.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880100f90 0x7f0880199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0880101950 0x7f0880199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880107a60 0x7f088019db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0880112df0 con 0x7f0880100f90 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0880112c70 con 0x7f0880107a60 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0880112f70 con 0x7f0880101950 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880100f90 0x7f0880199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880100f90 0x7f0880199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54870/0 (socket says 172.21.15.96:54870) 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 -- 172.21.15.96:0/3941515870 learned_addr learned my addr 172.21.15.96:0/3941515870 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0886767700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880107a60 0x7f088019db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885765700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0880101950 0x7f0880199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 -- 172.21.15.96:0/3941515870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0880101950 msgr2=0x7f0880199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.805 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0880101950 0x7f0880199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 -- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880107a60 msgr2=0x7f088019db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0880107a60 0x7f088019db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 -- 172.21.15.96:0/3941515870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088019e250 con 0x7f0880100f90 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f0885f66700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880100f90 0x7f0880199240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f087800ba50 tx=0x7f087800be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0878019070 con 0x7f0880100f90 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088019e540 con 0x7f0880100f90 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0878014440 con 0x7f0880100f90 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08780075d0 con 0x7f0880100f90 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.802+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08801a5dc0 con 0x7f0880100f90 2024-04-25T20:55:48.806 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.803+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0880063150 con 0x7f0880100f90 2024-04-25T20:55:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f087801c070 con 0x7f0880100f90 2024-04-25T20:55:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f087effd700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f08680a3cd0 0x7f08680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.807 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f0885765700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f08680a3cd0 0x7f08680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f08780d07b0 con 0x7f0880100f90 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f087effd700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] conn(0x7f08680ad710 0x7f08680afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 --> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f087800aea0 con 0x7f08680ad710 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.805+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f08780d0c10 con 0x7f0880100f90 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.806+0000 7f0885765700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f08680a3cd0 0x7f08680a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0874006fd0 tx=0x7f0874008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.806+0000 7f0886767700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] conn(0x7f08680ad710 0x7f08680afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.806+0000 7f0886767700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] conn(0x7f08680ad710 0x7f08680afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:48.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.807+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== osd.4 v2:172.21.15.97:6808/443367186 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f087800aea0 con 0x7f08680ad710 2024-04-25T20:55:48.824 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.823+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 --> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f08800635d0 con 0x7f08680ad710 2024-04-25T20:55:48.825 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.824+0000 7f087effd700 1 -- 172.21.15.96:0/3941515870 <== osd.4 v2:172.21.15.97:6808/443367186 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f08800635d0 con 0x7f08680ad710 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.824+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] conn(0x7f08680ad710 msgr2=0x7f08680afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.824+0000 7f08881ca700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] conn(0x7f08680ad710 0x7f08680afbb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f08680a3cd0 msgr2=0x7f08680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f08680a3cd0 0x7f08680a6190 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0874006fd0 tx=0x7f0874008040 comp rx=0 tx=0).stop 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880100f90 msgr2=0x7f0880199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 --2- 172.21.15.96:0/3941515870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0880100f90 0x7f0880199240 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f087800ba50 tx=0x7f087800be10 comp rx=0 tx=0).stop 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f0886767700 1 -- 172.21.15.96:0/3941515870 reap_dead start 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 shutdown_connections 2024-04-25T20:55:48.826 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 >> 172.21.15.96:0/3941515870 conn(0x7f0880077f40 msgr2=0x7f0880105a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:48.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 shutdown_connections 2024-04-25T20:55:48.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:48.825+0000 7f08881ca700 1 -- 172.21.15.96:0/3941515870 wait complete. 2024-04-25T20:55:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:48 smithi097 ceph-mon[32667]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:48 smithi139 ceph-mon[33027]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:49.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 -- 172.21.15.96:0/915750128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 msgr2=0x7f2dd010e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.021 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/915750128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd010e7f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2dc800b0a0 tx=0x7f2dc800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 -- 172.21.15.96:0/915750128 shutdown_connections 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/915750128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd010e7f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/915750128 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0107990 0x7f2dd0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/915750128 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101940 0x7f2dd0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 -- 172.21.15.96:0/915750128 >> 172.21.15.96:0/915750128 conn(0x7f2dd00788f0 msgr2=0x7f2dd0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 -- 172.21.15.96:0/915750128 shutdown_connections 2024-04-25T20:55:49.022 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.020+0000 7f2dd4db9700 1 -- 172.21.15.96:0/915750128 wait complete. 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 Processor -- start 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 -- start start 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101940 0x7f2dd01034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0107990 0x7f2dd0103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111af0 con 0x7f2dd0102300 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111970 con 0x7f2dd0107990 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dd4db9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd0111c70 con 0x7f2dd0101940 2024-04-25T20:55:49.024 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dcdd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dcdd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd0103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54890/0 (socket says 172.21.15.96:54890) 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dcdd9b700 1 -- 172.21.15.96:0/2113018657 learned_addr learned my addr 172.21.15.96:0/2113018657 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dce59c700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101940 0x7f2dd01034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dced9d700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0107990 0x7f2dd0103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.021+0000 7f2dcdd9b700 1 -- 172.21.15.96:0/2113018657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101940 msgr2=0x7f2dd01034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dcdd9b700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101940 0x7f2dd01034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dcdd9b700 1 -- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0107990 msgr2=0x7f2dd0103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dcdd9b700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0107990 0x7f2dd0103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dcdd9b700 1 -- 172.21.15.96:0/2113018657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd01a75c0 con 0x7f2dd0102300 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dce59c700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2dd0101940 0x7f2dd01034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dced9d700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2dd0107990 0x7f2dd0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dcdd9b700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd0103a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c970 tx=0x7f2dc400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:49.025 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4007aa0 con 0x7f2dd0102300 2024-04-25T20:55:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc4004ce0 con 0x7f2dd0102300 2024-04-25T20:55:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd01a78b0 con 0x7f2dd0102300 2024-04-25T20:55:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40055c0 con 0x7f2dd0102300 2024-04-25T20:55:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.022+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd01a7da0 con 0x7f2dd0102300 2024-04-25T20:55:49.026 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.023+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2dd00630d0 con 0x7f2dd0102300 2024-04-25T20:55:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.024+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2dc4007c40 con 0x7f2dd0102300 2024-04-25T20:55:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.024+0000 7f2dbf7fe700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2db80a3cd0 0x7f2db80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dce59c700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2db80a3cd0 0x7f2db80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f2dc40d0710 con 0x7f2dd0102300 2024-04-25T20:55:49.027 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dbf7fe700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] conn(0x7f2db80ad710 0x7f2db80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 --> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2dc400eec0 con 0x7f2db80ad710 2024-04-25T20:55:49.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f2dc40d0bf0 con 0x7f2dd0102300 2024-04-25T20:55:49.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dce59c700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2db80a3cd0 0x7f2db80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005e10 tx=0x7f2dc0005d60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:49.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dced9d700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] conn(0x7f2db80ad710 0x7f2db80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.025+0000 7f2dced9d700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] conn(0x7f2db80ad710 0x7f2db80afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:49.028 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.026+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== osd.7 v2:172.21.15.97:6816/2284407992 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2dc400eec0 con 0x7f2db80ad710 2024-04-25T20:55:49.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.042+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 --> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2dd00634d0 con 0x7f2db80ad710 2024-04-25T20:55:49.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dbf7fe700 1 -- 172.21.15.96:0/2113018657 <== osd.7 v2:172.21.15.97:6816/2284407992 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2dd00634d0 con 0x7f2db80ad710 2024-04-25T20:55:49.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] conn(0x7f2db80ad710 msgr2=0x7f2db80afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] conn(0x7f2db80ad710 0x7f2db80afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2db80a3cd0 msgr2=0x7f2db80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2db80a3cd0 0x7f2db80a6190 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0005e10 tx=0x7f2dc0005d60 comp rx=0 tx=0).stop 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 msgr2=0x7f2dd0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 --2- 172.21.15.96:0/2113018657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2dd0102300 0x7f2dd0103a30 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400c970 tx=0x7f2dc400cd30 comp rx=0 tx=0).stop 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dced9d700 1 -- 172.21.15.96:0/2113018657 reap_dead start 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 shutdown_connections 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 >> 172.21.15.96:0/2113018657 conn(0x7f2dd00788f0 msgr2=0x7f2dd006aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.043+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 shutdown_connections 2024-04-25T20:55:49.045 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:49.044+0000 7f2dd4db9700 1 -- 172.21.15.96:0/2113018657 wait complete. 2024-04-25T20:55:49.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:49 smithi096 ceph-mon[29288]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T20:55:49.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 -- 172.21.15.96:0/2016783234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0101940 msgr2=0x7fd7b0101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.556 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/2016783234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0101940 0x7fd7b0101dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd7a0009990 tx=0x7fd7a0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 -- 172.21.15.96:0/2016783234 shutdown_connections 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/2016783234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b0102300 0x7fd7b010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/2016783234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b0107990 0x7fd7b0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/2016783234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0101940 0x7fd7b0101dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 -- 172.21.15.96:0/2016783234 >> 172.21.15.96:0/2016783234 conn(0x7fd7b00788f0 msgr2=0x7fd7b0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 -- 172.21.15.96:0/2016783234 shutdown_connections 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.517+0000 7fd7b4bde700 1 -- 172.21.15.96:0/2016783234 wait complete. 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 Processor -- start 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 -- start start 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b0101940 0x7fd7b0197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b0102300 0x7fd7b0198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0107990 0x7fd7b01987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.557 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0111b50 con 0x7fd7b0107990 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b01119d0 con 0x7fd7b0101940 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7b4bde700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fd7b0111cd0 con 0x7fd7b0102300 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7aed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0107990 0x7fd7b01987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7aed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0107990 0x7fd7b01987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54806/0 (socket says 172.21.15.96:54806) 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7aed9d700 1 -- 172.21.15.96:0/3090355077 learned_addr learned my addr 172.21.15.96:0/3090355077 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7ae59c700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b0101940 0x7fd7b0197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.518+0000 7fd7add9b700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b0102300 0x7fd7b0198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7aed9d700 1 -- 172.21.15.96:0/3090355077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b0102300 msgr2=0x7fd7b0198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7aed9d700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fd7b0102300 0x7fd7b0198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7aed9d700 1 -- 172.21.15.96:0/3090355077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b0101940 msgr2=0x7fd7b0197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7aed9d700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7b0101940 0x7fd7b0197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7aed9d700 1 -- 172.21.15.96:0/3090355077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7b01a7590 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7aed9d700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0107990 0x7fd7b01987a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800b970 tx=0x7fd7a800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8019070 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7a800cde0 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7b01a7880 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8007370 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.519+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7b01a7e10 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.520+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd7b0063150 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.521+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd7a801c070 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.521+0000 7fd79f7fe700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd7980a3d50 0x7fd7980a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd7ae59c700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd7980a3d50 0x7fd7980a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fd7a80d01b0 con 0x7fd7b0107990 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd79f7fe700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] conn(0x7fd7980ad790 0x7fd7980afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 --> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd7980b0300 con 0x7fd7980ad790 2024-04-25T20:55:49.559 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fd7a80d0690 con 0x7fd7b0107990 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd7add9b700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] conn(0x7fd7980ad790 0x7fd7980afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd7ae59c700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd7980a3d50 0x7fd7980a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400ac50 tx=0x7fd7a400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.522+0000 7fd7add9b700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] conn(0x7fd7980ad790 0x7fd7980afc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.523+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== osd.6 v2:172.21.15.96:6818/262813972 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd7980b0300 con 0x7fd7980ad790 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.540+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 --> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd7b0063550 con 0x7fd7980ad790 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.540+0000 7fd79f7fe700 1 -- 172.21.15.96:0/3090355077 <== osd.6 v2:172.21.15.96:6818/262813972 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fd7b0063550 con 0x7fd7980ad790 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] conn(0x7fd7980ad790 msgr2=0x7fd7980afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] conn(0x7fd7980ad790 0x7fd7980afc30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd7980a3d50 msgr2=0x7fd7980a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fd7980a3d50 0x7fd7980a6210 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400ac50 tx=0x7fd7a400a380 comp rx=0 tx=0).stop 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0107990 msgr2=0x7fd7b01987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 --2- 172.21.15.96:0/3090355077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd7b0107990 0x7fd7b01987a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800b970 tx=0x7fd7a800bc80 comp rx=0 tx=0).stop 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7aed9d700 1 -- 172.21.15.96:0/3090355077 reap_dead start 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 shutdown_connections 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 >> 172.21.15.96:0/3090355077 conn(0x7fd7b00788f0 msgr2=0x7fd7b00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 shutdown_connections 2024-04-25T20:55:49.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.541+0000 7fd7b4bde700 1 -- 172.21.15.96:0/3090355077 wait complete. 2024-04-25T20:55:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:49 smithi097 ceph-mon[32667]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T20:55:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:49 smithi139 ceph-mon[33027]: pgmap v343: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 7 op/s 2024-04-25T20:55:50.253 INFO:teuthology.orchestra.run.smithi096.stdout:201863462971 2024-04-25T20:55:50.254 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/4179171337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 msgr2=0x7f1eb810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/4179171337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb810e7e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000b0a0 tx=0x7f1eb000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/4179171337 shutdown_connections 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/4179171337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb810e7e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/4179171337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb8107980 0x7f1eb8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/4179171337 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eb8101930 0x7f1eb8101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/4179171337 >> 172.21.15.96:0/4179171337 conn(0x7f1eb80788b0 msgr2=0x7f1eb8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/4179171337 shutdown_connections 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.450+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/4179171337 wait complete. 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 Processor -- start 2024-04-25T20:55:50.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 -- start start 2024-04-25T20:55:50.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb8101930 0x7f1eb8199070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb81995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eb8107980 0x7f1eb819d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8111ae0 con 0x7f1eb81022f0 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8111960 con 0x7f1eb8101930 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebf3e5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1eb8111c60 con 0x7f1eb8107980 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebc980700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb81995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebc980700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb81995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54900/0 (socket says 172.21.15.96:54900) 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebc980700 1 -- 172.21.15.96:0/1244920967 learned_addr learned my addr 172.21.15.96:0/1244920967 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.451+0000 7f1ebd181700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb8101930 0x7f1eb8199070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebd982700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eb8107980 0x7f1eb819d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebc980700 1 -- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eb8107980 msgr2=0x7f1eb819d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebc980700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eb8107980 0x7f1eb819d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebc980700 1 -- 172.21.15.96:0/1244920967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb8101930 msgr2=0x7f1eb8199070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebc980700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1eb8101930 0x7f1eb8199070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebc980700 1 -- 172.21.15.96:0/1244920967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1eb819e080 con 0x7f1eb81022f0 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebd982700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1eb8107980 0x7f1eb819d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebc980700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb81995b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800e9c0 tx=0x7f1ea800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800cbd0 con 0x7f1eb81022f0 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ea800cd70 con 0x7f1eb81022f0 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1eb819e370 con 0x7f1eb81022f0 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8005610 con 0x7f1eb81022f0 2024-04-25T20:55:50.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.452+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1eb81a5bf0 con 0x7f1eb81022f0 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.453+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1eb8063150 con 0x7f1eb81022f0 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.454+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1ea8004750 con 0x7f1eb81022f0 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.455+0000 7f1eae7fc700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1ea40a3d50 0x7f1ea40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.455+0000 7f1ebd181700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1ea40a3d50 0x7f1ea40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.455+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f1ea80d0b30 con 0x7f1eb81022f0 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.455+0000 7f1eae7fc700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] conn(0x7f1ea40ad790 0x7f1ea40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.455+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 --> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ea801ee50 con 0x7f1ea40ad790 2024-04-25T20:55:50.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.455+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f1ea80d1010 con 0x7f1eb81022f0 2024-04-25T20:55:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.456+0000 7f1ebd181700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1ea40a3d50 0x7f1ea40a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009be0 tx=0x7f1eb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.456+0000 7f1ebd982700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] conn(0x7f1ea40ad790 0x7f1ea40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.456+0000 7f1ebd982700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] conn(0x7f1ea40ad790 0x7f1ea40afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:50.458 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.457+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== osd.5 v2:172.21.15.139:6808/2385064558 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ea801ee50 con 0x7f1ea40ad790 2024-04-25T20:55:50.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.474+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 --> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1eb80635d0 con 0x7f1ea40ad790 2024-04-25T20:55:50.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.474+0000 7f1eae7fc700 1 -- 172.21.15.96:0/1244920967 <== osd.5 v2:172.21.15.139:6808/2385064558 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1eb80635d0 con 0x7f1ea40ad790 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.474+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] conn(0x7f1ea40ad790 msgr2=0x7f1ea40afc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.474+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] conn(0x7f1ea40ad790 0x7f1ea40afc30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1ea40a3d50 msgr2=0x7f1ea40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1ea40a3d50 0x7f1ea40a6210 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009be0 tx=0x7f1eb4009260 comp rx=0 tx=0).stop 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 msgr2=0x7f1eb81995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 --2- 172.21.15.96:0/1244920967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1eb81022f0 0x7f1eb81995b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800e9c0 tx=0x7f1ea800ecd0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebd982700 1 -- 172.21.15.96:0/1244920967 reap_dead start 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 shutdown_connections 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 >> 172.21.15.96:0/1244920967 conn(0x7f1eb80788b0 msgr2=0x7f1eb806b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 shutdown_connections 2024-04-25T20:55:50.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:50.475+0000 7f1ebf3e5700 1 -- 172.21.15.96:0/1244920967 wait complete. 2024-04-25T20:55:50.653 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 -- 172.21.15.96:0/882012706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 msgr2=0x7f9ff0108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/882012706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0108e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4009990 tx=0x7f9fe4009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 -- 172.21.15.96:0/882012706 shutdown_connections 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/882012706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff006b0f0 0x7f9ff010f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/882012706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0108e10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/882012706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff006a750 0x7f9ff006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 -- 172.21.15.96:0/882012706 >> 172.21.15.96:0/882012706 conn(0x7f9ff00788f0 msgr2=0x7f9ff0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 -- 172.21.15.96:0/882012706 shutdown_connections 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.629+0000 7f9ff6b52700 1 -- 172.21.15.96:0/882012706 wait complete. 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 Processor -- start 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 -- start start 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff006a750 0x7f9ff0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff006b0f0 0x7f9ff0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.654 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0112bb0 con 0x7f9ff0108a30 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0112a30 con 0x7f9ff006b0f0 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff6b52700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff0112d30 con 0x7f9ff006a750 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0102490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54824/0 (socket says 172.21.15.96:54824) 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 -- 172.21.15.96:0/2832034030 learned_addr learned my addr 172.21.15.96:0/2832034030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9feffff700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff006b0f0 0x7f9ff0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff48ee700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff006a750 0x7f9ff0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 -- 172.21.15.96:0/2832034030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff006a750 msgr2=0x7f9ff0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff006a750 0x7f9ff0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 -- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff006b0f0 msgr2=0x7f9ff0101f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff006b0f0 0x7f9ff0101f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.655 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 -- 172.21.15.96:0/2832034030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff01a76b0 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff48ee700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9ff006a750 0x7f9ff0101a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.630+0000 7f9ff50ef700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0102490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8007ee0 tx=0x7f9fe800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.631+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800ddd0 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.631+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff01a79a0 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.631+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fe8004020 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.631+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8015660 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.631+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff01a7ee0 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.632+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ff0063110 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.633+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9fe80041d0 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.633+0000 7f9fedffb700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9fe00a3cd0 0x7f9fe00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.633+0000 7f9ff48ee700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9fe00a3cd0 0x7f9fe00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.633+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f9fe80d09c0 con 0x7f9ff0108a30 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.634+0000 7f9fedffb700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] conn(0x7f9fe00ad710 0x7f9fe00afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:50.656 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.634+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 --> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9fe8025e50 con 0x7f9fe00ad710 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.634+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f9fe80d0ea0 con 0x7f9ff0108a30 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.634+0000 7f9ff48ee700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9fe00a3cd0 0x7f9fe00a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b2e0 tx=0x7f9fe4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.634+0000 7f9feffff700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] conn(0x7f9fe00ad710 0x7f9fe00afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.634+0000 7f9feffff700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] conn(0x7f9fe00ad710 0x7f9fe00afbb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.635+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== osd.10 v2:172.21.15.97:6824/3642406379 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9fe8025e50 con 0x7f9fe00ad710 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.651+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 --> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ff0063590 con 0x7f9fe00ad710 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9fedffb700 1 -- 172.21.15.96:0/2832034030 <== osd.10 v2:172.21.15.97:6824/3642406379 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9ff0063590 con 0x7f9fe00ad710 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] conn(0x7f9fe00ad710 msgr2=0x7f9fe00afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] conn(0x7f9fe00ad710 0x7f9fe00afbb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9fe00a3cd0 msgr2=0x7f9fe00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.657 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9fe00a3cd0 0x7f9fe00a6190 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9fe400b2e0 tx=0x7f9fe4005fd0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 msgr2=0x7f9ff0102490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 --2- 172.21.15.96:0/2832034030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ff0108a30 0x7f9ff0102490 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8007ee0 tx=0x7f9fe800d3b0 comp rx=0 tx=0).stop 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff50ef700 1 -- 172.21.15.96:0/2832034030 reap_dead start 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 shutdown_connections 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 >> 172.21.15.96:0/2832034030 conn(0x7f9ff00788f0 msgr2=0x7f9ff00ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 shutdown_connections 2024-04-25T20:55:50.658 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:47.652+0000 7f9ff6b52700 1 -- 172.21.15.96:0/2832034030 wait complete. 2024-04-25T20:55:51.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 -- 172.21.15.96:0/3414821267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28107990 msgr2=0x7fef28107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3414821267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28107990 0x7fef28107d70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fef18009990 tx=0x7fef18009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 -- 172.21.15.96:0/3414821267 shutdown_connections 2024-04-25T20:55:51.443 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3414821267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28102300 0x7fef2810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3414821267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28107990 0x7fef28107d70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3414821267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef28101940 0x7fef28101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 -- 172.21.15.96:0/3414821267 >> 172.21.15.96:0/3414821267 conn(0x7fef280788f0 msgr2=0x7fef28078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:51.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 -- 172.21.15.96:0/3414821267 shutdown_connections 2024-04-25T20:55:51.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.442+0000 7fef2d31f700 1 -- 172.21.15.96:0/3414821267 wait complete. 2024-04-25T20:55:51.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 Processor -- start 2024-04-25T20:55:51.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 -- start start 2024-04-25T20:55:51.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28101940 0x7fef28198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28102300 0x7fef28198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef28107990 0x7fef2819c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef28111ae0 con 0x7fef28102300 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fef28111960 con 0x7fef28101940 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef2d31f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fef28111c60 con 0x7fef28107990 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28102300 0x7fef28198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef1ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28102300 0x7fef28198570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54918/0 (socket says 172.21.15.96:54918) 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.443+0000 7fef1ffff700 1 -- 172.21.15.96:0/3008086297 learned_addr learned my addr 172.21.15.96:0/3008086297 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef277fe700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef28107990 0x7fef2819c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef26ffd700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28101940 0x7fef28198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef1ffff700 1 -- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef28107990 msgr2=0x7fef2819c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef1ffff700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fef28107990 0x7fef2819c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef1ffff700 1 -- 172.21.15.96:0/3008086297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28101940 msgr2=0x7fef28198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef1ffff700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef28101940 0x7fef28198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.446 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef1ffff700 1 -- 172.21.15.96:0/3008086297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef2819d040 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef1ffff700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28102300 0x7fef28198570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c920 tx=0x7fef1000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10004a30 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef100054d0 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef2819d330 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef10007c70 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.444+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef2806a720 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.445+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fef2806acf0 con 0x7fef28102300 2024-04-25T20:55:51.447 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.446+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fef10004c60 con 0x7fef28102300 2024-04-25T20:55:51.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef24ff9700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef140a8160 0x7fef140aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef26ffd700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef140a8160 0x7fef140aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fef100d0360 con 0x7fef28102300 2024-04-25T20:55:51.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef24ff9700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] conn(0x7fef140b1ba0 0x7fef140b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:51.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 --> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef10023e50 con 0x7fef140b1ba0 2024-04-25T20:55:51.448 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fef100d0710 con 0x7fef28102300 2024-04-25T20:55:51.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.447+0000 7fef26ffd700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef140a8160 0x7fef140aa620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b5c0 tx=0x7fef18005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:51.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.448+0000 7fef277fe700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] conn(0x7fef140b1ba0 0x7fef140b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:51.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.448+0000 7fef277fe700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] conn(0x7fef140b1ba0 0x7fef140b4040 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:51.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.449+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== osd.11 v2:172.21.15.139:6824/1534901897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef10023e50 con 0x7fef140b1ba0 2024-04-25T20:55:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.465+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 --> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fef2806b170 con 0x7fef140b1ba0 2024-04-25T20:55:51.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef24ff9700 1 -- 172.21.15.96:0/3008086297 <== osd.11 v2:172.21.15.139:6824/1534901897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fef2806b170 con 0x7fef140b1ba0 2024-04-25T20:55:51.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] conn(0x7fef140b1ba0 msgr2=0x7fef140b4040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] conn(0x7fef140b1ba0 0x7fef140b4040 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef140a8160 msgr2=0x7fef140aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fef140a8160 0x7fef140aa620 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fef1800b5c0 tx=0x7fef18005960 comp rx=0 tx=0).stop 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28102300 msgr2=0x7fef28198570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 --2- 172.21.15.96:0/3008086297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef28102300 0x7fef28198570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fef1000c920 tx=0x7fef1000cc30 comp rx=0 tx=0).stop 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef277fe700 1 -- 172.21.15.96:0/3008086297 reap_dead start 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 shutdown_connections 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 >> 172.21.15.96:0/3008086297 conn(0x7fef280788f0 msgr2=0x7fef28100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 shutdown_connections 2024-04-25T20:55:51.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:51.466+0000 7fef2d31f700 1 -- 172.21.15.96:0/3008086297 wait complete. 2024-04-25T20:55:51.567 INFO:teuthology.orchestra.run.smithi096.stdout:322122547238 2024-04-25T20:55:51.568 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-25T20:55:52.363 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:52 smithi096 ceph-mon[29288]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T20:55:52.399 INFO:teuthology.orchestra.run.smithi096.stdout:356482285602 2024-04-25T20:55:52.399 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-25T20:55:52.566 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:52 smithi097 ceph-mon[32667]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T20:55:52.575 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:52 smithi139 ceph-mon[33027]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T20:55:52.986 INFO:teuthology.orchestra.run.smithi096.stdout:34359738456 2024-04-25T20:55:52.986 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 -- 172.21.15.96:0/4025577820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4107990 msgr2=0x7f20f4107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 --2- 172.21.15.96:0/4025577820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4107990 0x7f20f4107d70 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f20e4009990 tx=0x7f20e4009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 -- 172.21.15.96:0/4025577820 shutdown_connections 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 --2- 172.21.15.96:0/4025577820 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f4102300 0x7f20f410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 --2- 172.21.15.96:0/4025577820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4107990 0x7f20f4107d70 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 --2- 172.21.15.96:0/4025577820 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f4101940 0x7f20f4101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 -- 172.21.15.96:0/4025577820 >> 172.21.15.96:0/4025577820 conn(0x7f20f40788f0 msgr2=0x7f20f4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:53.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 -- 172.21.15.96:0/4025577820 shutdown_connections 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 -- 172.21.15.96:0/4025577820 wait complete. 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 Processor -- start 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.270+0000 7f20fa410700 1 -- start start 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20fa410700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4101940 0x7f20f4199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20fa410700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f4102300 0x7f20f41997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20fa410700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f4107990 0x7f20f419daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20fa410700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4111bb0 con 0x7f20f4101940 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20fa410700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4111a30 con 0x7f20f4102300 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20fa410700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f20f4111d30 con 0x7f20f4107990 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4101940 0x7f20f4199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4101940 0x7f20f4199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54944/0 (socket says 172.21.15.96:54944) 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 -- 172.21.15.96:0/2509651498 learned_addr learned my addr 172.21.15.96:0/2509651498 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f37fe700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f4102300 0x7f20f41997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f89ad700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f4107990 0x7f20f419daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:53.279 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 -- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f4107990 msgr2=0x7f20f419daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f4107990 0x7f20f419daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 -- 172.21.15.96:0/2509651498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f4102300 msgr2=0x7f20f41997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20f4102300 0x7f20f41997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 -- 172.21.15.96:0/2509651498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f419e230 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f89ad700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f20f4107990 0x7f20f419daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.271+0000 7f20f3fff700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4101940 0x7f20f4199270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f20e4005910 tx=0x7f20e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.272+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4027070 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.272+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f419e4c0 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.272+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e4003f20 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.272+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4022c70 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.272+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f41a5da0 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.272+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20f4106950 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.274+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f20e402a070 con 0x7f20f4101940 2024-04-25T20:55:53.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.274+0000 7f20f17fa700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f20dc0a3cd0 0x7f20dc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.274+0000 7f20f37fe700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f20dc0a3cd0 0x7f20dc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.274+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f20e40d1d60 con 0x7f20f4101940 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.275+0000 7f20f17fa700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] conn(0x7f20dc0ad710 0x7f20dc0afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.275+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 --> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20dc0b0280 con 0x7f20dc0ad710 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.275+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f20e40d2240 con 0x7f20f4101940 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.275+0000 7f20f37fe700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f20dc0a3cd0 0x7f20dc0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20f4078ff0 tx=0x7f20e8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.275+0000 7f20f89ad700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] conn(0x7f20dc0ad710 0x7f20dc0afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.275+0000 7f20f89ad700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] conn(0x7f20dc0ad710 0x7f20dc0afbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:53.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.276+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== osd.8 v2:172.21.15.139:6816/3100320732 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20dc0b0280 con 0x7f20dc0ad710 2024-04-25T20:55:53.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.293+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 --> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20f4103060 con 0x7f20dc0ad710 2024-04-25T20:55:53.294 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20f17fa700 1 -- 172.21.15.96:0/2509651498 <== osd.8 v2:172.21.15.139:6816/3100320732 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20f4103060 con 0x7f20dc0ad710 2024-04-25T20:55:53.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] conn(0x7f20dc0ad710 msgr2=0x7f20dc0afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:53.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20fa410700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] conn(0x7f20dc0ad710 0x7f20dc0afbb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f20dc0a3cd0 msgr2=0x7f20dc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:53.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20fa410700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f20dc0a3cd0 0x7f20dc0a6190 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f20f4078ff0 tx=0x7f20e8009190 comp rx=0 tx=0).stop 2024-04-25T20:55:53.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4101940 msgr2=0x7f20f4199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:53.295 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20fa410700 1 --2- 172.21.15.96:0/2509651498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20f4101940 0x7f20f4199270 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f20e4005910 tx=0x7f20e40047b0 comp rx=0 tx=0).stop 2024-04-25T20:55:53.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.294+0000 7f20f89ad700 1 -- 172.21.15.96:0/2509651498 reap_dead start 2024-04-25T20:55:53.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.295+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 shutdown_connections 2024-04-25T20:55:53.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.295+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 >> 172.21.15.96:0/2509651498 conn(0x7f20f40788f0 msgr2=0x7f20f406a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:53.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.296+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 shutdown_connections 2024-04-25T20:55:53.296 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:53.296+0000 7f20fa410700 1 -- 172.21.15.96:0/2509651498 wait complete. 2024-04-25T20:55:54.135 INFO:teuthology.orchestra.run.smithi096.stdout:146028888131 2024-04-25T20:55:54.135 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-25T20:55:54.427 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:54 smithi096 ceph-mon[29288]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:54 smithi097 ceph-mon[32667]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:54 smithi139 ceph-mon[33027]: pgmap v345: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:55:54.770 INFO:teuthology.orchestra.run.smithi096.stdout:167503724607 2024-04-25T20:55:54.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-25T20:55:55.477 INFO:teuthology.orchestra.run.smithi096.stdout:296352743467 2024-04-25T20:55:55.477 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-25T20:55:55.919 INFO:teuthology.orchestra.run.smithi096.stdout:223338299444 2024-04-25T20:55:55.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-25T20:55:56.413 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:56 smithi096 ceph-mon[29288]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:56.442 INFO:teuthology.orchestra.run.smithi096.stdout:261993005104 2024-04-25T20:55:56.443 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-25T20:55:56.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 -- 172.21.15.96:0/993300174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94075590 msgr2=0x7f6f9410ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 --2- 172.21.15.96:0/993300174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94075590 0x7f6f9410ef00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c009990 tx=0x7f6f7c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 -- 172.21.15.96:0/993300174 shutdown_connections 2024-04-25T20:55:56.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 --2- 172.21.15.96:0/993300174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f9410f440 0x7f6f94111830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 --2- 172.21.15.96:0/993300174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f94074c70 0x7f6f94075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 --2- 172.21.15.96:0/993300174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94075590 0x7f6f9410ef00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 -- 172.21.15.96:0/993300174 >> 172.21.15.96:0/993300174 conn(0x7f6f94079ba0 msgr2=0x7f6f94079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 -- 172.21.15.96:0/993300174 shutdown_connections 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.107+0000 7f6f999fb700 1 -- 172.21.15.96:0/993300174 wait complete. 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 Processor -- start 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 -- start start 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94074c70 0x7f6f9419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f94075590 0x7f6f9419b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f9410f440 0x7f6f9419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94114e90 con 0x7f6f94074c70 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94114d10 con 0x7f6f9410f440 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f999fb700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6f94115010 con 0x7f6f94075590 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94074c70 0x7f6f9419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94074c70 0x7f6f9419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54964/0 (socket says 172.21.15.96:54964) 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 -- 172.21.15.96:0/2697340921 learned_addr learned my addr 172.21.15.96:0/2697340921 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f937fe700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f9410f440 0x7f6f9419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f8a7fc700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f94075590 0x7f6f9419b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 -- 172.21.15.96:0/2697340921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f94075590 msgr2=0x7f6f9419b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6f94075590 0x7f6f9419b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 -- 172.21.15.96:0/2697340921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f9410f440 msgr2=0x7f6f9419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6f9410f440 0x7f6f9419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 -- 172.21.15.96:0/2697340921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f941a0430 con 0x7f6f94074c70 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.108+0000 7f6f92ffd700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94074c70 0x7f6f9419b420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b530 tx=0x7f6f8400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.109+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84015020 con 0x7f6f94074c70 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.109+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f941a0720 con 0x7f6f94074c70 2024-04-25T20:55:56.532 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.109+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f84011ce0 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.109+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f84010d20 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.109+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f941a7fa0 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.110+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6f94063150 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.111+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6f84018020 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.111+0000 7f6f989f9700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f800a3cd0 0x7f6f800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.111+0000 7f6f8a7fc700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f800a3cd0 0x7f6f800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.111+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f6f840d0290 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.112+0000 7f6f989f9700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] conn(0x7f6f800ad710 0x7f6f800afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.112+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 --> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f8401deb0 con 0x7f6f800ad710 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.112+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7f6f840d0770 con 0x7f6f94074c70 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.112+0000 7f6f937fe700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] conn(0x7f6f800ad710 0x7f6f800afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:55:56.533 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.112+0000 7f6f8a7fc700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f800a3cd0 0x7f6f800a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c006010 tx=0x7f6f7c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.112+0000 7f6f937fe700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] conn(0x7f6f800ad710 0x7f6f800afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.113+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== osd.3 v2:172.21.15.96:6810/2644602517 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6f8401deb0 con 0x7f6f800ad710 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 --> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6f94063550 con 0x7f6f800ad710 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f989f9700 1 -- 172.21.15.96:0/2697340921 <== osd.3 v2:172.21.15.96:6810/2644602517 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6f94063550 con 0x7f6f800ad710 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] conn(0x7f6f800ad710 msgr2=0x7f6f800afbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] conn(0x7f6f800ad710 0x7f6f800afbb0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f800a3cd0 msgr2=0x7f6f800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6f800a3cd0 0x7f6f800a6190 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c006010 tx=0x7f6f7c01a040 comp rx=0 tx=0).stop 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94074c70 msgr2=0x7f6f9419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f999fb700 1 --2- 172.21.15.96:0/2697340921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f94074c70 0x7f6f9419b420 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6f8400b530 tx=0x7f6f8400b8f0 comp rx=0 tx=0).stop 2024-04-25T20:55:56.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.129+0000 7f6f937fe700 1 -- 172.21.15.96:0/2697340921 reap_dead start 2024-04-25T20:55:56.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.130+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 shutdown_connections 2024-04-25T20:55:56.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.130+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 >> 172.21.15.96:0/2697340921 conn(0x7f6f94079ba0 msgr2=0x7f6f941010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:55:56.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.130+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 shutdown_connections 2024-04-25T20:55:56.535 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:55:56.130+0000 7f6f999fb700 1 -- 172.21.15.96:0/2697340921 wait complete. 2024-04-25T20:55:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:56 smithi097 ceph-mon[32667]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:56 smithi139 ceph-mon[33027]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:55:56.771 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:56.771 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:56.775 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:57.195 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:57.203 INFO:teuthology.orchestra.run.smithi096.stdout:111669149770 2024-04-25T20:55:57.203 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-25T20:55:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:55:58 smithi097 ceph-mon[32667]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:55:58 smithi139 ceph-mon[33027]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:55:58 smithi096 ceph-mon[29288]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:55:59.970 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:59.974 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:59.975 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:55:59.984 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:00.469 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:00 smithi096 ceph-mon[29288]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:00 smithi097 ceph-mon[32667]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:00 smithi139 ceph-mon[33027]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:02 smithi097 ceph-mon[32667]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-25T20:56:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:02 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3593782756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T20:56:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:02 smithi139 ceph-mon[33027]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-25T20:56:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:02 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3593782756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T20:56:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:02 smithi096 ceph-mon[29288]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-04-25T20:56:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:02 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3593782756' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-25T20:56:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:04 smithi096 ceph-mon[29288]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T20:56:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:04 smithi097 ceph-mon[32667]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T20:56:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:04 smithi139 ceph-mon[33027]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-04-25T20:56:05.574 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.362+0000 7ffb6e731700 1 -- 172.21.15.96:0/1619466977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb681019d0 msgr2=0x7ffb6810f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.362+0000 7ffb6e731700 1 --2- 172.21.15.96:0/1619466977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb681019d0 0x7ffb6810f910 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000b0a0 tx=0x7ffb6000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 -- 172.21.15.96:0/1619466977 shutdown_connections 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 --2- 172.21.15.96:0/1619466977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb681019d0 0x7ffb6810f910 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 --2- 172.21.15.96:0/1619466977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb68108ad0 0x7ffb68108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 --2- 172.21.15.96:0/1619466977 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68101030 0x7ffb68101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 -- 172.21.15.96:0/1619466977 >> 172.21.15.96:0/1619466977 conn(0x7ffb68077fb0 msgr2=0x7ffb680ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 -- 172.21.15.96:0/1619466977 shutdown_connections 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 -- 172.21.15.96:0/1619466977 wait complete. 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 Processor -- start 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.363+0000 7ffb6e731700 1 -- start start 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68101030 0x7ffb68199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:05.575 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb681019d0 0x7ffb681995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 0x7ffb6819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68112be0 con 0x7ffb68108ad0 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68112a60 con 0x7ffb681019d0 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ffb68112d60 con 0x7ffb68101030 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 0x7ffb6819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 0x7ffb6819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58008/0 (socket says 172.21.15.96:58008) 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 -- 172.21.15.96:0/3593782756 learned_addr learned my addr 172.21.15.96:0/3593782756 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb67fff700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68101030 0x7ffb68199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb677fe700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb681019d0 0x7ffb681995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 -- 172.21.15.96:0/3593782756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68101030 msgr2=0x7ffb68199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68101030 0x7ffb68199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 -- 172.21.15.96:0/3593782756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb681019d0 msgr2=0x7ffb681995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb681019d0 0x7ffb681995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 -- 172.21.15.96:0/3593782756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb6819e0a0 con 0x7ffb68108ad0 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb677fe700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb681019d0 0x7ffb681995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6ccce700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 0x7ffb6819d960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffb600062a0 tx=0x7ffb60004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60021070 con 0x7ffb68108ad0 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb60007d00 con 0x7ffb68108ad0 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb6819e330 con 0x7ffb68108ad0 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stdout:167503724610 2024-04-25T20:56:05.577 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6001d450 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.364+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb681a5c10 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.366+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb68104350 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.367+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb60004020 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.367+0000 7ffb657fa700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffb500a3cd0 0x7ffb500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.367+0000 7ffb67fff700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffb500a3cd0 0x7ffb500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.367+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7ffb6000f020 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.368+0000 7ffb67fff700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffb500a3cd0 0x7ffb500a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005fd0 tx=0x7ffb5800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.372+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb6002a020 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.620+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7ffb6802df80 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.621+0000 7ffb657fa700 1 -- 172.21.15.96:0/3593782756 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffb6009e5d0 con 0x7ffb68108ad0 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffb500a3cd0 msgr2=0x7ffb500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:05.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffb500a3cd0 0x7ffb500a6190 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005fd0 tx=0x7ffb5800af20 comp rx=0 tx=0).stop 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 msgr2=0x7ffb6819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 0x7ffb6819d960 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffb600062a0 tx=0x7ffb60004ed0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 shutdown_connections 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ffb500a3cd0 0x7ffb500a6190 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb68108ad0 0x7ffb6819d960 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ffb68101030 0x7ffb68199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 --2- 172.21.15.96:0/3593782756 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffb681019d0 0x7ffb681995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 >> 172.21.15.96:0/3593782756 conn(0x7ffb68077fb0 msgr2=0x7ffb680ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 shutdown_connections 2024-04-25T20:56:05.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:01.623+0000 7ffb6e731700 1 -- 172.21.15.96:0/3593782756 wait complete. 2024-04-25T20:56:05.791 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:06.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:06 smithi096 ceph-mon[29288]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:06.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724607 got 167503724610 for osd.5 2024-04-25T20:56:06.566 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:06 smithi097 ceph-mon[32667]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:06 smithi139 ceph-mon[33027]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:07.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:07 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3583170855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T20:56:07.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:07 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3583170855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T20:56:07.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:07 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3583170855' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-25T20:56:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:08 smithi097 ceph-mon[32667]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:08 smithi139 ceph-mon[33027]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:08 smithi096 ceph-mon[29288]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/2654126013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e0108b10 msgr2=0x7fc6e0108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/2654126013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e0108b10 0x7fc6e0108ef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0099c0 tx=0x7fc6dc009cd0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/2654126013 shutdown_connections 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/2654126013 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e006bb50 0x7fc6e010f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/2654126013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e0108b10 0x7fc6e0108ef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/2654126013 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e006b190 0x7fc6e006b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.910 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/2654126013 >> 172.21.15.96:0/2654126013 conn(0x7fc6e0078830 msgr2=0x7fc6e0078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/2654126013 shutdown_connections 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.907+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/2654126013 wait complete. 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 Processor -- start 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 -- start start 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e006b190 0x7fc6e0199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 0x7fc6e0199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e0108b10 0x7fc6e019d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0112c10 con 0x7fc6e006bb50 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0112a90 con 0x7fc6e0108b10 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e7bbd700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc6e0112d90 con 0x7fc6e006b190 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5158700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 0x7fc6e0199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5158700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 0x7fc6e0199660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50570/0 (socket says 172.21.15.96:50570) 2024-04-25T20:56:08.911 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5158700 1 -- 172.21.15.96:0/4101364548 learned_addr learned my addr 172.21.15.96:0/4101364548 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5959700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e006b190 0x7fc6e0199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5158700 1 -- 172.21.15.96:0/4101364548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e006b190 msgr2=0x7fc6e0199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5158700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e006b190 0x7fc6e0199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e5158700 1 -- 172.21.15.96:0/4101364548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e0108b10 msgr2=0x7fc6e019d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.908+0000 7fc6e615a700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e0108b10 0x7fc6e019d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6e5158700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e0108b10 0x7fc6e019d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6e5158700 1 -- 172.21.15.96:0/4101364548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6e019e130 con 0x7fc6e006bb50 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6e5158700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 0x7fc6e0199660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c920 tx=0x7fc6d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0007b00 con 0x7fc6e006bb50 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6d0004ce0 con 0x7fc6e006bb50 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6e019e420 con 0x7fc6e006bb50 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6d0005650 con 0x7fc6e006bb50 2024-04-25T20:56:08.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.909+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6e01a5ca0 con 0x7fc6e006bb50 2024-04-25T20:56:08.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.910+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6e01041d0 con 0x7fc6e006bb50 2024-04-25T20:56:08.913 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.911+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc6d0004750 con 0x7fc6e006bb50 2024-04-25T20:56:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.912+0000 7fc6d6ffd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc6cc0a3cd0 0x7fc6cc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.912+0000 7fc6e5959700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc6cc0a3cd0 0x7fc6cc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.912+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fc6d00d05b0 con 0x7fc6e006bb50 2024-04-25T20:56:08.914 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.912+0000 7fc6e5959700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc6cc0a3cd0 0x7fc6cc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc003800 tx=0x7fc6dc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:08.916 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.916+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc6d0014030 con 0x7fc6e006bb50 2024-04-25T20:56:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 -- 172.21.15.96:0/1608144598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c102a90 msgr2=0x7f281c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 --2- 172.21.15.96:0/1608144598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c102a90 0x7f281c102f10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2810009990 tx=0x7f2810009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 -- 172.21.15.96:0/1608144598 shutdown_connections 2024-04-25T20:56:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 --2- 172.21.15.96:0/1608144598 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281c103450 0x7f281c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 --2- 172.21.15.96:0/1608144598 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c108a50 0x7f281c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.998 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 --2- 172.21.15.96:0/1608144598 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c102a90 0x7f281c102f10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 -- 172.21.15.96:0/1608144598 >> 172.21.15.96:0/1608144598 conn(0x7f281c079ba0 msgr2=0x7f281c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 -- 172.21.15.96:0/1608144598 shutdown_connections 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 -- 172.21.15.96:0/1608144598 wait complete. 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 Processor -- start 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.186+0000 7f2820c6d700 1 -- start start 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281c102a90 0x7f281c074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 0x7f281c075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c108a50 0x7f281c0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f281c112ce0 con 0x7f281c103450 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f281c112b60 con 0x7f281c108a50 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f281c112e60 con 0x7f281c102a90 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 0x7f281c075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 0x7f281c075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50546/0 (socket says 172.21.15.96:50546) 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 -- 172.21.15.96:0/2759780637 learned_addr learned my addr 172.21.15.96:0/2759780637 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:08.999 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f281a59c700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281c102a90 0x7f281c074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f281ad9d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c108a50 0x7f281c0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stdout:55834574935 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 -- 172.21.15.96:0/2759780637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281c102a90 msgr2=0x7f281c074d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281c102a90 0x7f281c074d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 -- 172.21.15.96:0/2759780637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c108a50 msgr2=0x7f281c0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c108a50 0x7f281c0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.000 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 -- 172.21.15.96:0/2759780637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f281c1a77e0 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2819d9b700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 0x7f281c075280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2810000c00 tx=0x7f28100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810027070 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2810003fe0 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f281c1a7a70 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2810022e30 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.187+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f281c1a7f50 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.188+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f281c069730 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.189+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f281002a070 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.190+0000 7f280b7fe700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28040a3cd0 0x7f28040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.190+0000 7f281a59c700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28040a3cd0 0x7f28040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.190+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f28100d11c0 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.190+0000 7f281a59c700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28040a3cd0 0x7f28040a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f280c009a10 tx=0x7f280c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.195+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2810013070 con 0x7f281c103450 2024-04-25T20:56:09.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.404+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f281c02e140 con 0x7f281c103450 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.405+0000 7f280b7fe700 1 -- 172.21.15.96:0/2759780637 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f281009dcf0 con 0x7f281c103450 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28040a3cd0 msgr2=0x7f28040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28040a3cd0 0x7f28040a6190 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f280c009a10 tx=0x7f280c009190 comp rx=0 tx=0).stop 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 msgr2=0x7f281c075280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 0x7f281c075280 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2810000c00 tx=0x7f28100047b0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 shutdown_connections 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28040a3cd0 0x7f28040a6190 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f281c108a50 0x7f281c0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f281c102a90 0x7f281c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 --2- 172.21.15.96:0/2759780637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f281c103450 0x7f281c075280 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 >> 172.21.15.96:0/2759780637 conn(0x7f281c079ba0 msgr2=0x7f281c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 shutdown_connections 2024-04-25T20:56:09.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:08.407+0000 7f2820c6d700 1 -- 172.21.15.96:0/2759780637 wait complete. 2024-04-25T20:56:09.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.117+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fc6e002df80 con 0x7fc6e006bb50 2024-04-25T20:56:09.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.118+0000 7fc6d6ffd700 1 -- 172.21.15.96:0/4101364548 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fc6d00cf090 con 0x7fc6e006bb50 2024-04-25T20:56:09.119 INFO:teuthology.orchestra.run.smithi096.stdout:322122547242 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc6cc0a3cd0 msgr2=0x7fc6cc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc6cc0a3cd0 0x7fc6cc0a6190 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc003800 tx=0x7fc6dc01a040 comp rx=0 tx=0).stop 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 msgr2=0x7fc6e0199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 0x7fc6e0199660 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc6d000c920 tx=0x7fc6d000cce0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 shutdown_connections 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc6cc0a3cd0 0x7fc6cc0a6190 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc6e0108b10 0x7fc6e019d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc6e006b190 0x7fc6e0199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 --2- 172.21.15.96:0/4101364548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc6e006bb50 0x7fc6e0199660 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 >> 172.21.15.96:0/4101364548 conn(0x7fc6e0078830 msgr2=0x7fc6e01009c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:09.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 shutdown_connections 2024-04-25T20:56:09.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.120+0000 7fc6e7bbd700 1 -- 172.21.15.96:0/4101364548 wait complete. 2024-04-25T20:56:09.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:09 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2759780637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T20:56:09.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:09 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4101364548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T20:56:09.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:09 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2759780637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T20:56:09.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:09 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4101364548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T20:56:09.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 -- 172.21.15.96:0/1989501166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0107960 msgr2=0x7f61c0107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.618 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1989501166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0107960 0x7f61c0107d40 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009990 tx=0x7f61b0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 -- 172.21.15.96:0/1989501166 shutdown_connections 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1989501166 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61c0102320 0x7f61c010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1989501166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0107960 0x7f61c0107d40 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1989501166 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c0101960 0x7f61c0101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 -- 172.21.15.96:0/1989501166 >> 172.21.15.96:0/1989501166 conn(0x7f61c00789e0 msgr2=0x7f61c0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 -- 172.21.15.96:0/1989501166 shutdown_connections 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.617+0000 7f61c57eb700 1 -- 172.21.15.96:0/1989501166 wait complete. 2024-04-25T20:56:09.619 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 Processor -- start 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 -- start start 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 0x7f61c0198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c0102320 0x7f61c0198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61c0107960 0x7f61c019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111ba0 con 0x7f61c0101960 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111a20 con 0x7f61c0102320 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61c57eb700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f61c0111d20 con 0x7f61c0107960 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61beffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 0x7f61c0198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:09.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61beffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 0x7f61c0198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50578/0 (socket says 172.21.15.96:50578) 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.618+0000 7f61beffd700 1 -- 172.21.15.96:0/1807120518 learned_addr learned my addr 172.21.15.96:0/1807120518 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61be7fc700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c0102320 0x7f61c0198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61beffd700 1 -- 172.21.15.96:0/1807120518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61c0107960 msgr2=0x7f61c019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61beffd700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61c0107960 0x7f61c019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61beffd700 1 -- 172.21.15.96:0/1807120518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c0102320 msgr2=0x7f61c0198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61beffd700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c0102320 0x7f61c0198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61beffd700 1 -- 172.21.15.96:0/1807120518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61c019d040 con 0x7f61c0101960 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61beffd700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 0x7f61c0198030 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000c00 tx=0x7f61b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0027070 con 0x7f61c0101960 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61c019d2d0 con 0x7f61c0101960 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61b0003fa0 con 0x7f61c0101960 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61b0022db0 con 0x7f61c0101960 2024-04-25T20:56:09.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.619+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61c01a4bb0 con 0x7f61c0101960 2024-04-25T20:56:09.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.620+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61c0109f60 con 0x7f61c0101960 2024-04-25T20:56:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:09 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2759780637' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-25T20:56:09.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:09 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4101364548' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-25T20:56:09.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.621+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f61b002a070 con 0x7f61c0101960 2024-04-25T20:56:09.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.622+0000 7f61a7fff700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f61a80a3cd0 0x7f61a80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:09.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.622+0000 7f61be7fc700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f61a80a3cd0 0x7f61a80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:09.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.622+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f61b00d1550 con 0x7f61c0101960 2024-04-25T20:56:09.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.622+0000 7f61be7fc700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f61a80a3cd0 0x7f61a80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005fd0 tx=0x7f61b4005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:09.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.625+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61b0013070 con 0x7f61c0101960 2024-04-25T20:56:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.835+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f61c0199390 con 0x7f61c0101960 2024-04-25T20:56:09.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.835+0000 7f61a7fff700 1 -- 172.21.15.96:0/1807120518 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f61b009df60 con 0x7f61c0101960 2024-04-25T20:56:09.836 INFO:teuthology.orchestra.run.smithi096.stdout:261993005107 2024-04-25T20:56:09.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f61a80a3cd0 msgr2=0x7f61a80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f61a80a3cd0 0x7f61a80a6190 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f61b4005fd0 tx=0x7f61b4005f60 comp rx=0 tx=0).stop 2024-04-25T20:56:09.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 msgr2=0x7f61c0198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:09.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 0x7f61c0198030 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f61b0000c00 tx=0x7f61b00047b0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 shutdown_connections 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f61a80a3cd0 0x7f61a80a6190 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f61c0107960 0x7f61c019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61c0101960 0x7f61c0198030 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 --2- 172.21.15.96:0/1807120518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61c0102320 0x7f61c0198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 >> 172.21.15.96:0/1807120518 conn(0x7f61c00789e0 msgr2=0x7f61c0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 shutdown_connections 2024-04-25T20:56:09.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:09.837+0000 7f61c57eb700 1 -- 172.21.15.96:0/1807120518 wait complete. 2024-04-25T20:56:10.200 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547238 got 322122547242 for osd.10 2024-04-25T20:56:10.200 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:10 smithi096 ceph-mon[29288]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:10.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:10 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1807120518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T20:56:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:10 smithi097 ceph-mon[32667]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:10 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1807120518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T20:56:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:10 smithi139 ceph-mon[33027]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:10 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1807120518' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-25T20:56:10.872 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005104 got 261993005107 for osd.8 2024-04-25T20:56:10.872 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:10.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 -- 172.21.15.96:0/2042939298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d80102af0 msgr2=0x7f9d80102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:10.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 --2- 172.21.15.96:0/2042939298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d80102af0 0x7f9d80102f70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9d740099c0 tx=0x7f9d74009cd0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 -- 172.21.15.96:0/2042939298 shutdown_connections 2024-04-25T20:56:10.960 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 --2- 172.21.15.96:0/2042939298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d801034b0 0x7f9d8010f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 --2- 172.21.15.96:0/2042939298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80107940 0x7f9d80107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 --2- 172.21.15.96:0/2042939298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d80102af0 0x7f9d80102f70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 -- 172.21.15.96:0/2042939298 >> 172.21.15.96:0/2042939298 conn(0x7f9d80079ba0 msgr2=0x7f9d80079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 -- 172.21.15.96:0/2042939298 shutdown_connections 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 -- 172.21.15.96:0/2042939298 wait complete. 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.531+0000 7f9d86874700 1 Processor -- start 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 -- start start 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80102af0 0x7f9d801991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 0x7f9d801996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d80107940 0x7f9d8019da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80112cb0 con 0x7f9d801034b0 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80112b30 con 0x7f9d80102af0 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d86874700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80112e30 con 0x7f9d80107940 2024-04-25T20:56:10.961 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 0x7f9d801996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 0x7f9d801996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50534/0 (socket says 172.21.15.96:50534) 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 -- 172.21.15.96:0/3583170855 learned_addr learned my addr 172.21.15.96:0/3583170855 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7ffff700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80102af0 0x7f9d801991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d84e11700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d80107940 0x7f9d8019da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 -- 172.21.15.96:0/3583170855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d80107940 msgr2=0x7f9d8019da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d80107940 0x7f9d8019da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 -- 172.21.15.96:0/3583170855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80102af0 msgr2=0x7f9d801991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80102af0 0x7f9d801991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7f7fe700 1 -- 172.21.15.96:0/3583170855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d8019e1b0 con 0x7f9d801034b0 2024-04-25T20:56:10.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.532+0000 7f9d7ffff700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80102af0 0x7f9d801991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stdout:77309411409 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.533+0000 7f9d7f7fe700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 0x7f9d801996e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400b580 tx=0x7f9d740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.533+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74027070 con 0x7f9d801034b0 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.533+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8019e440 con 0x7f9d801034b0 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.533+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d74004020 con 0x7f9d801034b0 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.533+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74022e30 con 0x7f9d801034b0 2024-04-25T20:56:10.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.533+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d801a5d20 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.534+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d80069730 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.535+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9d74022590 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.536+0000 7f9d7d7fa700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d680a3cd0 0x7f9d680a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.536+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f9d740d15f0 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.536+0000 7f9d7ffff700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d680a3cd0 0x7f9d680a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.536+0000 7f9d7ffff700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d680a3cd0 0x7f9d680a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d700099d0 tx=0x7f9d70008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.540+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9d74030070 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.748+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f9d8002df80 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.748+0000 7f9d7d7fa700 1 -- 172.21.15.96:0/3583170855 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9d74013070 con 0x7f9d801034b0 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.750+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d680a3cd0 msgr2=0x7f9d680a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:10.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.750+0000 7f9d86874700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d680a3cd0 0x7f9d680a6190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d700099d0 tx=0x7f9d70008040 comp rx=0 tx=0).stop 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.750+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 msgr2=0x7f9d801996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.750+0000 7f9d86874700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 0x7f9d801996e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9d7400b580 tx=0x7f9d740047b0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 shutdown_connections 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f9d680a3cd0 0x7f9d680a6190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9d80107940 0x7f9d8019da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d80102af0 0x7f9d801991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 --2- 172.21.15.96:0/3583170855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d801034b0 0x7f9d801996e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 >> 172.21.15.96:0/3583170855 conn(0x7f9d80079ba0 msgr2=0x7f9d80100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 shutdown_connections 2024-04-25T20:56:10.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:06.751+0000 7f9d86874700 1 -- 172.21.15.96:0/3583170855 wait complete. 2024-04-25T20:56:11.160 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:11.161 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:11.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:11 smithi097 ceph-mon[32667]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T20:56:11.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:11 smithi139 ceph-mon[33027]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T20:56:11.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:11 smithi096 ceph-mon[29288]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-25T20:56:11.678 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411405 got 77309411409 for osd.2 2024-04-25T20:56:11.678 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:12 smithi097 ceph-mon[32667]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:56:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:12 smithi139 ceph-mon[33027]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:56:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:12 smithi096 ceph-mon[29288]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:56:12.811 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574928 got 55834574935 for osd.1 2024-04-25T20:56:12.811 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:14.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:14 smithi139 ceph-mon[33027]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:56:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:14 smithi096 ceph-mon[29288]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:56:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:14 smithi097 ceph-mon[32667]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.895+0000 7fee3c0db700 1 -- 172.21.15.96:0/3474651164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406b190 msgr2=0x7fee3406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.895+0000 7fee3c0db700 1 --2- 172.21.15.96:0/3474651164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406b190 0x7fee3406b5f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fee24009990 tx=0x7fee24009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- 172.21.15.96:0/3474651164 shutdown_connections 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 --2- 172.21.15.96:0/3474651164 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee3406bb30 0x7fee3410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 --2- 172.21.15.96:0/3474651164 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee34108a30 0x7fee34108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 --2- 172.21.15.96:0/3474651164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406b190 0x7fee3406b5f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- 172.21.15.96:0/3474651164 >> 172.21.15.96:0/3474651164 conn(0x7fee34078910 msgr2=0x7fee34078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- 172.21.15.96:0/3474651164 shutdown_connections 2024-04-25T20:56:14.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- 172.21.15.96:0/3474651164 wait complete. 2024-04-25T20:56:14.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 Processor -- start 2024-04-25T20:56:14.899 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- start start 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3406b190 0x7fee34199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 0x7fee341996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee34108a30 0x7fee3419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee34112c30 con 0x7fee3406bb30 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fee34112ab0 con 0x7fee34108a30 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.896+0000 7fee3c0db700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fee34112db0 con 0x7fee3406b190 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 0x7fee341996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 0x7fee341996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50592/0 (socket says 172.21.15.96:50592) 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 -- 172.21.15.96:0/2624335145 learned_addr learned my addr 172.21.15.96:0/2624335145 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39e77700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3406b190 0x7fee34199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee3a678700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee34108a30 0x7fee3419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:14.900 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 -- 172.21.15.96:0/2624335145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3406b190 msgr2=0x7fee34199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3406b190 0x7fee34199180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 -- 172.21.15.96:0/2624335145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee34108a30 msgr2=0x7fee3419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee34108a30 0x7fee3419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 -- 172.21.15.96:0/2624335145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3419e190 con 0x7fee3406bb30 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee3a678700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee34108a30 0x7fee3419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39e77700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3406b190 0x7fee34199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee39676700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 0x7fee341996c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fee240057c0 tx=0x7fee240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24027070 con 0x7fee3406bb30 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee24004020 con 0x7fee3406bb30 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24022bf0 con 0x7fee3406bb30 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3419e420 con 0x7fee3406bb30 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.897+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee341a5d00 con 0x7fee3406bb30 2024-04-25T20:56:14.901 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.898+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee34069730 con 0x7fee3406bb30 2024-04-25T20:56:14.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.899+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fee2402a070 con 0x7fee3406bb30 2024-04-25T20:56:14.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.900+0000 7fee2affd700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee200a3c40 0x7fee200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:14.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.900+0000 7fee39e77700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee200a3c40 0x7fee200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:14.902 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.900+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fee240d1280 con 0x7fee3406bb30 2024-04-25T20:56:14.903 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.900+0000 7fee39e77700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee200a3c40 0x7fee200a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee300099d0 tx=0x7fee30008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:14.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:14.904+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee2405eff0 con 0x7fee3406bb30 2024-04-25T20:56:15.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/4208114639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78103450 msgr2=0x7f2b7810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/4208114639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78103450 0x7f2b7810f8e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000b0a0 tx=0x7f2b7000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/4208114639 shutdown_connections 2024-04-25T20:56:15.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/4208114639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78103450 0x7f2b7810f8e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.106 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/4208114639 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b78108a80 0x7f2b78108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/4208114639 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b78102a90 0x7f2b78102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/4208114639 >> 172.21.15.96:0/4208114639 conn(0x7f2b78079bf0 msgr2=0x7f2b7807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:15.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/4208114639 shutdown_connections 2024-04-25T20:56:15.107 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.105+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/4208114639 wait complete. 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 Processor -- start 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 -- start start 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b78102a90 0x7f2b7819b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b78103450 0x7f2b7819b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 0x7f2b7819fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78112c10 con 0x7f2b78108a80 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78112a90 con 0x7f2b78102a90 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7d1fa700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2b78112d90 con 0x7f2b78103450 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 0x7f2b7819fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 0x7f2b7819fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50606/0 (socket says 172.21.15.96:50606) 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.106+0000 7f2b7759e700 1 -- 172.21.15.96:0/157406619 learned_addr learned my addr 172.21.15.96:0/157406619 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:15.109 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b76d9d700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b78102a90 0x7f2b7819b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7659c700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b78103450 0x7f2b7819b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7759e700 1 -- 172.21.15.96:0/157406619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b78103450 msgr2=0x7f2b7819b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7759e700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b78103450 0x7f2b7819b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7759e700 1 -- 172.21.15.96:0/157406619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b78102a90 msgr2=0x7f2b7819b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7759e700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b78102a90 0x7f2b7819b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7759e700 1 -- 172.21.15.96:0/157406619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b781a0280 con 0x7f2b78108a80 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7759e700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 0x7f2b7819fb40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000b780 tx=0x7f2b70004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70021070 con 0x7f2b78108a80 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b781a0510 con 0x7f2b78108a80 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2b70007e30 con 0x7f2b78108a80 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700045b0 con 0x7f2b78108a80 2024-04-25T20:56:15.110 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.107+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b781a7df0 con 0x7f2b78108a80 2024-04-25T20:56:15.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.109+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b78069730 con 0x7f2b78108a80 2024-04-25T20:56:15.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.109+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fee3402df80 con 0x7fee3406bb30 2024-04-25T20:56:15.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.110+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2b70004750 con 0x7f2b78108a80 2024-04-25T20:56:15.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.110+0000 7fee2affd700 1 -- 172.21.15.96:0/2624335145 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fee24013070 con 0x7fee3406bb30 2024-04-25T20:56:15.112 INFO:teuthology.orchestra.run.smithi096.stdout:223338299449 2024-04-25T20:56:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.110+0000 7f2b5ffff700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2b600a3c40 0x7f2b600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.110+0000 7f2b76d9d700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2b600a3c40 0x7f2b600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.110+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f2b7000f020 con 0x7f2b78108a80 2024-04-25T20:56:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.111+0000 7f2b76d9d700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2b600a3c40 0x7f2b600a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009be0 tx=0x7f2b6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee200a3c40 msgr2=0x7fee200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee200a3c40 0x7fee200a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fee300099d0 tx=0x7fee30008040 comp rx=0 tx=0).stop 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 msgr2=0x7fee341996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 0x7fee341996c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fee240057c0 tx=0x7fee240057f0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 shutdown_connections 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fee200a3c40 0x7fee200a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fee34108a30 0x7fee3419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fee3406b190 0x7fee34199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 --2- 172.21.15.96:0/2624335145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee3406bb30 0x7fee341996c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 >> 172.21.15.96:0/2624335145 conn(0x7fee34078910 msgr2=0x7fee340ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 shutdown_connections 2024-04-25T20:56:15.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.112+0000 7fee3c0db700 1 -- 172.21.15.96:0/2624335145 wait complete. 2024-04-25T20:56:15.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.115+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b7002a020 con 0x7f2b78108a80 2024-04-25T20:56:15.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.319+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f2b781a7fc0 con 0x7f2b78108a80 2024-04-25T20:56:15.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.319+0000 7f2b5ffff700 1 -- 172.21.15.96:0/157406619 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2b7009e1a0 con 0x7f2b78108a80 2024-04-25T20:56:15.320 INFO:teuthology.orchestra.run.smithi096.stdout:34359738461 2024-04-25T20:56:15.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2b600a3c40 msgr2=0x7f2b600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2b600a3c40 0x7f2b600a6100 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009be0 tx=0x7f2b6c009260 comp rx=0 tx=0).stop 2024-04-25T20:56:15.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 msgr2=0x7f2b7819fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:15.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 0x7f2b7819fb40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000b780 tx=0x7f2b70004d00 comp rx=0 tx=0).stop 2024-04-25T20:56:15.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 shutdown_connections 2024-04-25T20:56:15.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f2b600a3c40 0x7f2b600a6100 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2b78108a80 0x7f2b7819fb40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b78102a90 0x7f2b7819b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 --2- 172.21.15.96:0/157406619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2b78103450 0x7f2b7819b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:15.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 >> 172.21.15.96:0/157406619 conn(0x7f2b78079bf0 msgr2=0x7f2b7810dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:15.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 shutdown_connections 2024-04-25T20:56:15.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.321+0000 7f2b7d1fa700 1 -- 172.21.15.96:0/157406619 wait complete. 2024-04-25T20:56:15.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:15 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2624335145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T20:56:15.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:15 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/157406619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T20:56:15.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:15 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2624335145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T20:56:15.676 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:15 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/157406619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T20:56:15.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:15 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2624335145' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-25T20:56:15.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:15 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/157406619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-25T20:56:16.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 -- 172.21.15.96:0/3625088935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 msgr2=0x7feb60102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 --2- 172.21.15.96:0/3625088935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60102f10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7feb50009990 tx=0x7feb50009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 -- 172.21.15.96:0/3625088935 shutdown_connections 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 --2- 172.21.15.96:0/3625088935 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb60103450 0x7feb6010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 --2- 172.21.15.96:0/3625088935 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb60108a50 0x7feb60108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 --2- 172.21.15.96:0/3625088935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60102f10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 -- 172.21.15.96:0/3625088935 >> 172.21.15.96:0/3625088935 conn(0x7feb60079ba0 msgr2=0x7feb60079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 -- 172.21.15.96:0/3625088935 shutdown_connections 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.775+0000 7feb6786b700 1 -- 172.21.15.96:0/3625088935 wait complete. 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 Processor -- start 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 -- start start 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb60103450 0x7feb60199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:16.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb60108a50 0x7feb6019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb60112be0 con 0x7feb60102a90 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7feb60112a60 con 0x7feb60103450 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb6786b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7feb60112d60 con 0x7feb60108a50 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb65607700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb65607700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50614/0 (socket says 172.21.15.96:50614) 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb65607700 1 -- 172.21.15.96:0/1490146594 learned_addr learned my addr 172.21.15.96:0/1490146594 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.776+0000 7feb64e06700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb60103450 0x7feb60199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65e08700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb60108a50 0x7feb6019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65607700 1 -- 172.21.15.96:0/1490146594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb60108a50 msgr2=0x7feb6019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65607700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb60108a50 0x7feb6019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65607700 1 -- 172.21.15.96:0/1490146594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb60103450 msgr2=0x7feb60199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65607700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb60103450 0x7feb60199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65607700 1 -- 172.21.15.96:0/1490146594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb6019e160 con 0x7feb60102a90 2024-04-25T20:56:16.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb65607700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60199150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c920 tx=0x7feb5c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c004a30 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb5c0054d0 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5c007c70 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb6019e450 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.777+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb601a5cd0 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.778+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb60069730 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.780+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feb5c004c60 con 0x7feb60102a90 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.781+0000 7feb567fc700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb4c0a3c40 0x7feb4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.781+0000 7feb64e06700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb4c0a3c40 0x7feb4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:16.190 INFO:teuthology.orchestra.run.smithi096.stdout:201863462976 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.781+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7feb5c0d0180 con 0x7feb60102a90 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.781+0000 7feb64e06700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb4c0a3c40 0x7feb4c0a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb500098e0 tx=0x7feb50005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.785+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb5c014030 con 0x7feb60102a90 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.989+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7feb6019a4b0 con 0x7feb60102a90 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.990+0000 7feb567fc700 1 -- 172.21.15.96:0/1490146594 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feb5c09cc00 con 0x7feb60102a90 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb4c0a3c40 msgr2=0x7feb4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb4c0a3c40 0x7feb4c0a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7feb500098e0 tx=0x7feb50005960 comp rx=0 tx=0).stop 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 msgr2=0x7feb60199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60199150 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00c920 tx=0x7feb5c00cc30 comp rx=0 tx=0).stop 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 shutdown_connections 2024-04-25T20:56:16.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7feb4c0a3c40 0x7feb4c0a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7feb60108a50 0x7feb6019da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb60102a90 0x7feb60199150 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 --2- 172.21.15.96:0/1490146594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7feb60103450 0x7feb60199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:16.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 >> 172.21.15.96:0/1490146594 conn(0x7feb60079ba0 msgr2=0x7feb60101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:16.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 shutdown_connections 2024-04-25T20:56:16.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:15.992+0000 7feb6786b700 1 -- 172.21.15.96:0/1490146594 wait complete. 2024-04-25T20:56:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:16 smithi096 ceph-mon[29288]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T20:56:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:16 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1490146594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T20:56:16.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:16 smithi097 ceph-mon[32667]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T20:56:16.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:16 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1490146594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T20:56:16.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:16 smithi139 ceph-mon[33027]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T20:56:16.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:16 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1490146594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-25T20:56:16.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462971 got 201863462976 for osd.6 2024-04-25T20:56:16.917 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:17.804 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299444 got 223338299449 for osd.7 2024-04-25T20:56:17.805 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:18.478 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:18 smithi096 ceph-mon[29288]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:56:18.819 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738461 for osd.0 2024-04-25T20:56:18.819 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:18.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:18 smithi097 ceph-mon[32667]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:56:18.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:18 smithi139 ceph-mon[33027]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:56:20.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 -- 172.21.15.96:0/2701835367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01054b0 msgr2=0x7ff3f0105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:20.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 --2- 172.21.15.96:0/2701835367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01054b0 0x7ff3f0105910 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff3e800b0a0 tx=0x7ff3e800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 -- 172.21.15.96:0/2701835367 shutdown_connections 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 --2- 172.21.15.96:0/2701835367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01054b0 0x7ff3f0105910 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 --2- 172.21.15.96:0/2701835367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f01079a0 0x7ff3f006b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 --2- 172.21.15.96:0/2701835367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f006b730 0x7ff3f006bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 -- 172.21.15.96:0/2701835367 >> 172.21.15.96:0/2701835367 conn(0x7ff3f00789e0 msgr2=0x7ff3f0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 -- 172.21.15.96:0/2701835367 shutdown_connections 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 -- 172.21.15.96:0/2701835367 wait complete. 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 Processor -- start 2024-04-25T20:56:20.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.338+0000 7ff3f6941700 1 -- start start 2024-04-25T20:56:20.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f6941700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f006b730 0x7ff3f019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:20.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f6941700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f01054b0 0x7ff3f019fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:20.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f6941700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 0x7ff3f01a3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f6941700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f0119220 con 0x7ff3f01079a0 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f6941700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01190a0 con 0x7ff3f006b730 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f6941700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff3f01193a0 con 0x7ff3f01054b0 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 0x7ff3f01a3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 0x7ff3f01a3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35040/0 (socket says 172.21.15.96:35040) 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 -- 172.21.15.96:0/3673772424 learned_addr learned my addr 172.21.15.96:0/3673772424 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3effff700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f006b730 0x7ff3f019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3ef7fe700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f01054b0 0x7ff3f019fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 -- 172.21.15.96:0/3673772424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f01054b0 msgr2=0x7ff3f019fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f01054b0 0x7ff3f019fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 -- 172.21.15.96:0/3673772424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f006b730 msgr2=0x7ff3f019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f006b730 0x7ff3f019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 -- 172.21.15.96:0/3673772424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3f01a45a0 con 0x7ff3f01079a0 2024-04-25T20:56:20.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3ef7fe700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f01054b0 0x7ff3f019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.339+0000 7ff3f4ede700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 0x7ff3f01a3e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80062a0 tx=0x7ff3e8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.340+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e8021070 con 0x7ff3f01079a0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.340+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e8007b80 con 0x7ff3f01079a0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.340+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e80052a0 con 0x7ff3f01079a0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.340+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3f01a4830 con 0x7ff3f01079a0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.340+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3f01ac110 con 0x7ff3f01079a0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.340+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3f0069730 con 0x7ff3f01079a0 2024-04-25T20:56:20.344 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.342+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff3e8024070 con 0x7ff3f01079a0 2024-04-25T20:56:20.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.342+0000 7ff3ed7fa700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff3d80a3cc0 0x7ff3d80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:20.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.342+0000 7ff3effff700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff3d80a3cc0 0x7ff3d80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:20.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.342+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7ff3e800f020 con 0x7ff3f01079a0 2024-04-25T20:56:20.345 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.343+0000 7ff3effff700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff3d80a3cc0 0x7ff3d80a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005950 tx=0x7ff3e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:20.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.346+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3e802b020 con 0x7ff3f01079a0 2024-04-25T20:56:20.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.550+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff3f01ac2e0 con 0x7ff3f01079a0 2024-04-25T20:56:20.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.550+0000 7ff3ed7fa700 1 -- 172.21.15.96:0/3673772424 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff3e809e7c0 con 0x7ff3f01079a0 2024-04-25T20:56:20.552 INFO:teuthology.orchestra.run.smithi096.stdout:356482285608 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.552+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff3d80a3cc0 msgr2=0x7ff3d80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.552+0000 7ff3f6941700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff3d80a3cc0 0x7ff3d80a6180 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005950 tx=0x7ff3e400b410 comp rx=0 tx=0).stop 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.552+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 msgr2=0x7ff3f01a3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.552+0000 7ff3f6941700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 0x7ff3f01a3e60 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff3e80062a0 tx=0x7ff3e8004ce0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 shutdown_connections 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff3d80a3cc0 0x7ff3d80a6180 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3f01079a0 0x7ff3f01a3e60 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff3f006b730 0x7ff3f019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 --2- 172.21.15.96:0/3673772424 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff3f01054b0 0x7ff3f019fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 >> 172.21.15.96:0/3673772424 conn(0x7ff3f00789e0 msgr2=0x7ff3f0104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 shutdown_connections 2024-04-25T20:56:20.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:20.553+0000 7ff3f6941700 1 -- 172.21.15.96:0/3673772424 wait complete. 2024-04-25T20:56:20.619 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:20 smithi096 ceph-mon[29288]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T20:56:20.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:20 smithi097 ceph-mon[32667]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T20:56:20.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:20 smithi139 ceph-mon[33027]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-04-25T20:56:21.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 -- 172.21.15.96:0/1198375502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101dc0 msgr2=0x7f506c10f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:21.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 --2- 172.21.15.96:0/1198375502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101dc0 0x7f506c10f830 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f506400b0a0 tx=0x7f506400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 -- 172.21.15.96:0/1198375502 shutdown_connections 2024-04-25T20:56:21.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 --2- 172.21.15.96:0/1198375502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101dc0 0x7f506c10f830 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 --2- 172.21.15.96:0/1198375502 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c107b20 0x7f506c107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.043 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 --2- 172.21.15.96:0/1198375502 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f506c101400 0x7f506c101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.041+0000 7f5073e88700 1 -- 172.21.15.96:0/1198375502 >> 172.21.15.96:0/1198375502 conn(0x7f506c076c60 msgr2=0x7f506c077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:21.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 -- 172.21.15.96:0/1198375502 shutdown_connections 2024-04-25T20:56:21.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 -- 172.21.15.96:0/1198375502 wait complete. 2024-04-25T20:56:21.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 Processor -- start 2024-04-25T20:56:21.044 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 -- start start 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 0x7f506c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c101dc0 0x7f506c19b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f506c107b20 0x7f506c19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f506c112cd0 con 0x7f506c101400 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f506c112b50 con 0x7f506c101dc0 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.042+0000 7f5073e88700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f506c112e50 con 0x7f506c107b20 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 0x7f506c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 0x7f506c19b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35070/0 (socket says 172.21.15.96:35070) 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 -- 172.21.15.96:0/3791839269 learned_addr learned my addr 172.21.15.96:0/3791839269 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:21.046 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5072425700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f506c107b20 0x7f506c19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071423700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c101dc0 0x7f506c19b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 -- 172.21.15.96:0/3791839269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f506c107b20 msgr2=0x7f506c19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f506c107b20 0x7f506c19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 -- 172.21.15.96:0/3791839269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c101dc0 msgr2=0x7f506c19b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c101dc0 0x7f506c19b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 -- 172.21.15.96:0/3791839269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f506c1a0340 con 0x7f506c101400 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071423700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c101dc0 0x7f506c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5071c24700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 0x7f506c19b330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f506800b500 tx=0x7f506800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068015020 con 0x7f506c101400 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f506c1a0630 con 0x7f506c101400 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5068011ce0 con 0x7f506c101400 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068010d20 con 0x7f506c101400 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.043+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f506c1a7eb0 con 0x7f506c101400 2024-04-25T20:56:21.047 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.044+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f506c069730 con 0x7f506c101400 2024-04-25T20:56:21.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.045+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5068018020 con 0x7f506c101400 2024-04-25T20:56:21.048 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.046+0000 7f5062ffd700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f50580a3c40 0x7f50580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.046+0000 7f5071423700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f50580a3c40 0x7f50580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.046+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f50680d0020 con 0x7f506c101400 2024-04-25T20:56:21.049 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.046+0000 7f5071423700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f50580a3c40 0x7f50580a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f506c107850 tx=0x7f505c00b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:21.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.050+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f506801f020 con 0x7f506c101400 2024-04-25T20:56:21.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.250+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f506c19c690 con 0x7f506c101400 2024-04-25T20:56:21.251 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.250+0000 7f5062ffd700 1 -- 172.21.15.96:0/3791839269 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f506c19c690 con 0x7f506c101400 2024-04-25T20:56:21.252 INFO:teuthology.orchestra.run.smithi096.stdout:296352743473 2024-04-25T20:56:21.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f50580a3c40 msgr2=0x7f50580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:21.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f50580a3c40 0x7f50580a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f506c107850 tx=0x7f505c00b2c0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 msgr2=0x7f506c19b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 0x7f506c19b330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f506800b500 tx=0x7f506800b810 comp rx=0 tx=0).stop 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 shutdown_connections 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f50580a3c40 0x7f50580a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f506c107b20 0x7f506c19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f506c101400 0x7f506c19b330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 --2- 172.21.15.96:0/3791839269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f506c101dc0 0x7f506c19b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 >> 172.21.15.96:0/3791839269 conn(0x7f506c076c60 msgr2=0x7f506c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:21.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 shutdown_connections 2024-04-25T20:56:21.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:21.252+0000 7f5073e88700 1 -- 172.21.15.96:0/3791839269 wait complete. 2024-04-25T20:56:21.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:21 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3673772424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T20:56:21.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:21 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3791839269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T20:56:21.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:21 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3673772424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T20:56:21.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:21 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3791839269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T20:56:21.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:21 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3673772424' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-25T20:56:21.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:21 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3791839269' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-25T20:56:22.102 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743467 got 296352743473 for osd.9 2024-04-25T20:56:22.102 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 -- 172.21.15.96:0/988728552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368107a60 msgr2=0x7f5368107e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 --2- 172.21.15.96:0/988728552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368107a60 0x7f5368107e40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f536000b0d0 tx=0x7f536000b3e0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 -- 172.21.15.96:0/988728552 shutdown_connections 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 --2- 172.21.15.96:0/988728552 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5368101950 0x7f536810f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 --2- 172.21.15.96:0/988728552 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368107a60 0x7f5368107e40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 --2- 172.21.15.96:0/988728552 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5368100f90 0x7f5368101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.544+0000 7f536cf00700 1 -- 172.21.15.96:0/988728552 >> 172.21.15.96:0/988728552 conn(0x7f5368077f40 msgr2=0x7f53680ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 -- 172.21.15.96:0/988728552 shutdown_connections 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 -- 172.21.15.96:0/988728552 wait complete. 2024-04-25T20:56:22.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 Processor -- start 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 -- start start 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5368100f90 0x7f5368199290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 0x7f53681997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5368107a60 0x7f536819db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5368112dc0 con 0x7f5368101950 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5368112c40 con 0x7f5368107a60 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.545+0000 7f536cf00700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5368112f40 con 0x7f5368100f90 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 0x7f53681997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 0x7f53681997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35092/0 (socket says 172.21.15.96:35092) 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 -- 172.21.15.96:0/3786676892 learned_addr learned my addr 172.21.15.96:0/3786676892 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f536659c700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5368100f90 0x7f5368199290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:22.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5366d9d700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5368107a60 0x7f536819db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 -- 172.21.15.96:0/3786676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5368100f90 msgr2=0x7f5368199290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5368100f90 0x7f5368199290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 -- 172.21.15.96:0/3786676892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5368107a60 msgr2=0x7f536819db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5368107a60 0x7f536819db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 -- 172.21.15.96:0/3786676892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536819e250 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f5365d9b700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 0x7f53681997d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f535c00b530 tx=0x7f535c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c015020 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f535c011ce0 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536819e540 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c010e40 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.546+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53681a5dc0 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.547+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f536810afe0 con 0x7f5368101950 2024-04-25T20:56:22.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.548+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f535c018020 con 0x7f5368101950 2024-04-25T20:56:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.549+0000 7f53577fe700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53500a3c40 0x7f53500a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.549+0000 7f536659c700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53500a3c40 0x7f53500a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.549+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f535c0d0130 con 0x7f5368101950 2024-04-25T20:56:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.549+0000 7f536659c700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53500a3c40 0x7f53500a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5360009fd0 tx=0x7f536000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:22.564 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.552+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f535c01f020 con 0x7f5368101950 2024-04-25T20:56:22.738 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:22 smithi096 ceph-mon[29288]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:56:22.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.757+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f536819a5a0 con 0x7f5368101950 2024-04-25T20:56:22.758 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.758+0000 7f53577fe700 1 -- 172.21.15.96:0/3786676892 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f535c09cbb0 con 0x7f5368101950 2024-04-25T20:56:22.759 INFO:teuthology.orchestra.run.smithi096.stdout:146028888138 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53500a3c40 msgr2=0x7f53500a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53500a3c40 0x7f53500a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f5360009fd0 tx=0x7f536000b040 comp rx=0 tx=0).stop 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 msgr2=0x7f53681997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 0x7f53681997d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f535c00b530 tx=0x7f535c00b840 comp rx=0 tx=0).stop 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 shutdown_connections 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f53500a3c40 0x7f53500a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5368107a60 0x7f536819db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5368100f90 0x7f5368199290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 --2- 172.21.15.96:0/3786676892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5368101950 0x7f53681997d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:22.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 >> 172.21.15.96:0/3786676892 conn(0x7f5368077f40 msgr2=0x7f5368105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:22.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 shutdown_connections 2024-04-25T20:56:22.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:22.760+0000 7f536cf00700 1 -- 172.21.15.96:0/3786676892 wait complete. 2024-04-25T20:56:22.764 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:22 smithi097 ceph-mon[32667]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:56:22.774 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:22 smithi139 ceph-mon[33027]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-04-25T20:56:23.089 INFO:tasks.cephadm.ceph_manager.ceph:need seq 356482285602 got 356482285608 for osd.11 2024-04-25T20:56:23.089 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:23.610 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:23 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3786676892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T20:56:23.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:23 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3786676892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T20:56:23.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:23 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3786676892' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-25T20:56:23.920 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888131 got 146028888138 for osd.4 2024-04-25T20:56:23.920 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 -- 172.21.15.96:0/2772678744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 msgr2=0x7f6c7810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/2772678744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c7810e7f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6c7000b0a0 tx=0x7f6c7000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 -- 172.21.15.96:0/2772678744 shutdown_connections 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/2772678744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c7810e7f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/2772678744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c78107920 0x7f6c78107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/2772678744 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c78101970 0x7f6c78101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 -- 172.21.15.96:0/2772678744 >> 172.21.15.96:0/2772678744 conn(0x7f6c78078910 msgr2=0x7f6c78078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 -- 172.21.15.96:0/2772678744 shutdown_connections 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 -- 172.21.15.96:0/2772678744 wait complete. 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 Processor -- start 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 -- start start 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.188+0000 7f6c7ef23700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c78101970 0x7f6c781034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ef23700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c78103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ef23700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c78107920 0x7f6c78103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ef23700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78111b60 con 0x7f6c78102330 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ef23700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6c781119e0 con 0x7f6c78101970 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ef23700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6c78111ce0 con 0x7f6c78107920 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c78103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c78103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35112/0 (socket says 172.21.15.96:35112) 2024-04-25T20:56:24.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 -- 172.21.15.96:0/3036977388 learned_addr learned my addr 172.21.15.96:0/3036977388 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ccbf700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c78101970 0x7f6c781034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7d4c0700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c78107920 0x7f6c78103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 -- 172.21.15.96:0/3036977388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c78107920 msgr2=0x7f6c78103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c78107920 0x7f6c78103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 -- 172.21.15.96:0/3036977388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c78101970 msgr2=0x7f6c781034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c78101970 0x7f6c781034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 -- 172.21.15.96:0/3036977388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c781a7600 con 0x7f6c78102330 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6ffff700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c78103a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b530 tx=0x7f6c6800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68015020 con 0x7f6c78102330 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c68011ce0 con 0x7f6c78102330 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68010dd0 con 0x7f6c78102330 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.189+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c781a78f0 con 0x7f6c78102330 2024-04-25T20:56:24.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.190+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c781a7de0 con 0x7f6c78102330 2024-04-25T20:56:24.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.191+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c78069730 con 0x7f6c78102330 2024-04-25T20:56:24.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.191+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6c68018020 con 0x7f6c78102330 2024-04-25T20:56:24.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.192+0000 7f6c6dffb700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c600a3c40 0x7f6c600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:24.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.192+0000 7f6c7ccbf700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c600a3c40 0x7f6c600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:24.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.192+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f6c680cfa90 con 0x7f6c78102330 2024-04-25T20:56:24.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.193+0000 7f6c7ccbf700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c600a3c40 0x7f6c600a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c74005950 tx=0x7f6c7400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:24.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.197+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c6801f020 con 0x7f6c78102330 2024-04-25T20:56:24.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.396+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6c7802df80 con 0x7f6c78102330 2024-04-25T20:56:24.399 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.398+0000 7f6c6dffb700 1 -- 172.21.15.96:0/3036977388 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6c6809c510 con 0x7f6c78102330 2024-04-25T20:56:24.399 INFO:teuthology.orchestra.run.smithi096.stdout:111669149775 2024-04-25T20:56:24.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c600a3c40 msgr2=0x7f6c600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:24.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c600a3c40 0x7f6c600a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6c74005950 tx=0x7f6c7400a300 comp rx=0 tx=0).stop 2024-04-25T20:56:24.401 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 msgr2=0x7f6c78103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c78103a30 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800b530 tx=0x7f6c6800b840 comp rx=0 tx=0).stop 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 shutdown_connections 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6c600a3c40 0x7f6c600a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6c78107920 0x7f6c78103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6c78101970 0x7f6c781034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 --2- 172.21.15.96:0/3036977388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6c78102330 0x7f6c78103a30 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 >> 172.21.15.96:0/3036977388 conn(0x7f6c78078910 msgr2=0x7f6c7806b470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.400+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 shutdown_connections 2024-04-25T20:56:24.402 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:24.401+0000 7f6c7ef23700 1 -- 172.21.15.96:0/3036977388 wait complete. 2024-04-25T20:56:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:24 smithi096 ceph-mon[29288]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T20:56:24.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:24 smithi097 ceph-mon[32667]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T20:56:24.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:24 smithi139 ceph-mon[33027]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 341 B/s wr, 5 op/s 2024-04-25T20:56:25.068 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149770 got 111669149775 for osd.3 2024-04-25T20:56:25.068 DEBUG:teuthology.parallel:result is None 2024-04-25T20:56:25.069 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T20:56:25.069 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:56:25.308 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:25.677 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:25 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3036977388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T20:56:25.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:25 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3036977388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T20:56:25.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:25 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3036977388' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-25T20:56:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:26 smithi096 ceph-mon[29288]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-25T20:56:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:26 smithi097 ceph-mon[32667]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-25T20:56:26.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:26 smithi139 ceph-mon[33027]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 -- 172.21.15.96:0/3942067827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49881022f0 msgr2=0x7f498810e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 --2- 172.21.15.96:0/3942067827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49881022f0 0x7f498810e7e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f498000b0a0 tx=0x7f498000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 -- 172.21.15.96:0/3942067827 shutdown_connections 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 --2- 172.21.15.96:0/3942067827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49881022f0 0x7f498810e7e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 --2- 172.21.15.96:0/3942067827 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4988107980 0x7f4988107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 --2- 172.21.15.96:0/3942067827 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4988101930 0x7f4988101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 -- 172.21.15.96:0/3942067827 >> 172.21.15.96:0/3942067827 conn(0x7f49880788b0 msgr2=0x7f4988078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 -- 172.21.15.96:0/3942067827 shutdown_connections 2024-04-25T20:56:27.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 -- 172.21.15.96:0/3942067827 wait complete. 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 Processor -- start 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.080+0000 7f498ebb5700 1 -- start start 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 0x7f498819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49881022f0 0x7f498819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4988107980 0x7f498819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4988111ad0 con 0x7f4988101930 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4988111950 con 0x7f4988107980 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4988111c50 con 0x7f49881022f0 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 0x7f498819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f4987fff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49881022f0 0x7f498819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 0x7f498819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50610/0 (socket says 172.21.15.96:50610) 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 -- 172.21.15.96:0/431493188 learned_addr learned my addr 172.21.15.96:0/431493188 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498d152700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4988107980 0x7f498819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:27.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 -- 172.21.15.96:0/431493188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49881022f0 msgr2=0x7f498819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49881022f0 0x7f498819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 -- 172.21.15.96:0/431493188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4988107980 msgr2=0x7f498819fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4988107980 0x7f498819fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 -- 172.21.15.96:0/431493188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49881a02c0 con 0x7f4988101930 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498c951700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 0x7f498819b2b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d950 tx=0x7f497c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c016070 con 0x7f4988101930 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49881a05b0 con 0x7f4988101930 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.081+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f497c00fd40 con 0x7f4988101930 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.082+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497c00e3b0 con 0x7f4988101930 2024-04-25T20:56:27.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.082+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49881a7e30 con 0x7f4988101930 2024-04-25T20:56:27.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.083+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4988069730 con 0x7f4988101930 2024-04-25T20:56:27.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.084+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f497c019070 con 0x7f4988101930 2024-04-25T20:56:27.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.084+0000 7f4985ffb700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f49780a3c40 0x7f49780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:27.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.084+0000 7f4987fff700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f49780a3c40 0x7f49780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:27.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.085+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f497c0d0170 con 0x7f4988101930 2024-04-25T20:56:27.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.085+0000 7f4987fff700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f49780a3c40 0x7f49780a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f498810eb90 tx=0x7f497400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:27.089 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.089+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f497c09cbd0 con 0x7f4988101930 2024-04-25T20:56:27.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.307+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f498806b6c0 con 0x7f49780a3c40 2024-04-25T20:56:27.314 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.314+0000 7f4985ffb700 1 -- 172.21.15.96:0/431493188 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258838 (secure 0 0 0) 0x7f498806b6c0 con 0x7f49780a3c40 2024-04-25T20:56:27.318 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f49780a3c40 msgr2=0x7f49780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f49780a3c40 0x7f49780a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f498810eb90 tx=0x7f497400b410 comp rx=0 tx=0).stop 2024-04-25T20:56:27.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 msgr2=0x7f498819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 0x7f498819b2b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d950 tx=0x7f497c00dc60 comp rx=0 tx=0).stop 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 shutdown_connections 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f49780a3c40 0x7f49780a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4988107980 0x7f498819fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4988101930 0x7f498819b2b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 --2- 172.21.15.96:0/431493188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f49881022f0 0x7f498819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 >> 172.21.15.96:0/431493188 conn(0x7f49880788b0 msgr2=0x7f498806afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 shutdown_connections 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:27.317+0000 7f498ebb5700 1 -- 172.21.15.96:0/431493188 wait complete. 2024-04-25T20:56:27.321 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:56:27.658 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:27 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:27.658 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:27 smithi096 ceph-mon[29288]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:27.658 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:27 smithi096 ceph-mon[29288]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:56:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:27 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:27 smithi097 ceph-mon[32667]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:27 smithi097 ceph-mon[32667]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:56:27.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:27 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:56:27.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:27 smithi139 ceph-mon[33027]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:27.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:27 smithi139 ceph-mon[33027]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:56:28.326 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-25T20:56:27.265500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":751,"num_read_kb":492,"num_write":354,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388092,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553732,"statfs":{"total":1151940427776,"available":1151543021568,"internally_reserved":0,"allocated":14315520,"data_stored":7901376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002777"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267515+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:56:09.267515+0000","last_peered":"2024-04-25T20:56:09.267515+0000","last_clean":"2024-04-25T20:56:09.267515+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:56:09.267515+0000","last_undegraded":"2024-04-25T20:56:09.267515+0000","last_fullsized":"2024-04-25T20:56:09.267515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256093+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:56:09.256093+0000","last_peered":"2024-04-25T20:56:09.256093+0000","last_clean":"2024-04-25T20:56:09.256093+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:56:09.256093+0000","last_undegraded":"2024-04-25T20:56:09.256093+0000","last_fullsized":"2024-04-25T20:56:09.256093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268636+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:56:09.268636+0000","last_peered":"2024-04-25T20:56:09.268636+0000","last_clean":"2024-04-25T20:56:09.268636+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:56:09.268636+0000","last_undegraded":"2024-04-25T20:56:09.268636+0000","last_fullsized":"2024-04-25T20:56:09.268636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257720+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:56:09.257720+0000","last_peered":"2024-04-25T20:56:09.257720+0000","last_clean":"2024-04-25T20:56:09.257720+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:56:09.257720+0000","last_undegraded":"2024-04-25T20:56:09.257720+0000","last_fullsized":"2024-04-25T20:56:09.257720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.247668+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:56:09.247668+0000","last_peered":"2024-04-25T20:56:09.247668+0000","last_clean":"2024-04-25T20:56:09.247668+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:56:09.247668+0000","last_undegraded":"2024-04-25T20:56:09.247668+0000","last_fullsized":"2024-04-25T20:56:09.247668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255680+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:56:09.255680+0000","last_peered":"2024-04-25T20:56:09.255680+0000","last_clean":"2024-04-25T20:56:09.255680+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:56:09.255680+0000","last_undegraded":"2024-04-25T20:56:09.255680+0000","last_fullsized":"2024-04-25T20:56:09.255680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.245845+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:56:09.245845+0000","last_peered":"2024-04-25T20:56:09.245845+0000","last_clean":"2024-04-25T20:56:09.245845+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:56:09.245845+0000","last_undegraded":"2024-04-25T20:56:09.245845+0000","last_fullsized":"2024-04-25T20:56:09.245845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.249193+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:56:09.249193+0000","last_peered":"2024-04-25T20:56:09.249193+0000","last_clean":"2024-04-25T20:56:09.249193+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:56:09.249193+0000","last_undegraded":"2024-04-25T20:56:09.249193+0000","last_fullsized":"2024-04-25T20:56:09.249193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.265574+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:56:09.265574+0000","last_peered":"2024-04-25T20:56:09.265574+0000","last_clean":"2024-04-25T20:56:09.265574+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:56:09.265574+0000","last_undegraded":"2024-04-25T20:56:09.265574+0000","last_fullsized":"2024-04-25T20:56:09.265574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257485+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:56:09.257485+0000","last_peered":"2024-04-25T20:56:09.257485+0000","last_clean":"2024-04-25T20:56:09.257485+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:56:09.257485+0000","last_undegraded":"2024-04-25T20:56:09.257485+0000","last_fullsized":"2024-04-25T20:56:09.257485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267062+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:56:09.267062+0000","last_peered":"2024-04-25T20:56:09.267062+0000","last_clean":"2024-04-25T20:56:09.267062+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:56:09.267062+0000","last_undegraded":"2024-04-25T20:56:09.267062+0000","last_fullsized":"2024-04-25T20:56:09.267062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255163+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:56:09.255163+0000","last_peered":"2024-04-25T20:56:09.255163+0000","last_clean":"2024-04-25T20:56:09.255163+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:56:09.255163+0000","last_undegraded":"2024-04-25T20:56:09.255163+0000","last_fullsized":"2024-04-25T20:56:09.255163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253188+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:56:09.253188+0000","last_peered":"2024-04-25T20:56:09.253188+0000","last_clean":"2024-04-25T20:56:09.253188+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:56:09.253188+0000","last_undegraded":"2024-04-25T20:56:09.253188+0000","last_fullsized":"2024-04-25T20:56:09.253188+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256936+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:56:09.256936+0000","last_peered":"2024-04-25T20:56:09.256936+0000","last_clean":"2024-04-25T20:56:09.256936+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:56:09.256936+0000","last_undegraded":"2024-04-25T20:56:09.256936+0000","last_fullsized":"2024-04-25T20:56:09.256936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255076+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:56:09.255076+0000","last_peered":"2024-04-25T20:56:09.255076+0000","last_clean":"2024-04-25T20:56:09.255076+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:56:09.255076+0000","last_undegraded":"2024-04-25T20:56:09.255076+0000","last_fullsized":"2024-04-25T20:56:09.255076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267239+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:56:09.267239+0000","last_peered":"2024-04-25T20:56:09.267239+0000","last_clean":"2024-04-25T20:56:09.267239+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:56:09.267239+0000","last_undegraded":"2024-04-25T20:56:09.267239+0000","last_fullsized":"2024-04-25T20:56:09.267239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267728+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:56:09.267728+0000","last_peered":"2024-04-25T20:56:09.267728+0000","last_clean":"2024-04-25T20:56:09.267728+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:56:09.267728+0000","last_undegraded":"2024-04-25T20:56:09.267728+0000","last_fullsized":"2024-04-25T20:56:09.267728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.258888+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:56:09.258888+0000","last_peered":"2024-04-25T20:56:09.258888+0000","last_clean":"2024-04-25T20:56:09.258888+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:56:09.258888+0000","last_undegraded":"2024-04-25T20:56:09.258888+0000","last_fullsized":"2024-04-25T20:56:09.258888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.254234+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:56:09.254234+0000","last_peered":"2024-04-25T20:56:09.254234+0000","last_clean":"2024-04-25T20:56:09.254234+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:56:09.254234+0000","last_undegraded":"2024-04-25T20:56:09.254234+0000","last_fullsized":"2024-04-25T20:56:09.254234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331587+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:55:10.331587+0000","last_peered":"2024-04-25T20:55:10.331587+0000","last_clean":"2024-04-25T20:55:10.331587+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:55:10.331587+0000","last_undegraded":"2024-04-25T20:55:10.331587+0000","last_fullsized":"2024-04-25T20:55:10.331587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255040+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:56:09.255040+0000","last_peered":"2024-04-25T20:56:09.255040+0000","last_clean":"2024-04-25T20:56:09.255040+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:56:09.255040+0000","last_undegraded":"2024-04-25T20:56:09.255040+0000","last_fullsized":"2024-04-25T20:56:09.255040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331113+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:55:10.331113+0000","last_peered":"2024-04-25T20:55:10.331113+0000","last_clean":"2024-04-25T20:55:10.331113+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:55:10.331113+0000","last_undegraded":"2024-04-25T20:55:10.331113+0000","last_fullsized":"2024-04-25T20:55:10.331113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269311+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:56:09.269311+0000","last_peered":"2024-04-25T20:56:09.269311+0000","last_clean":"2024-04-25T20:56:09.269311+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:56:09.269311+0000","last_undegraded":"2024-04-25T20:56:09.269311+0000","last_fullsized":"2024-04-25T20:56:09.269311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332839+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:55:10.332839+0000","last_peered":"2024-04-25T20:55:10.332839+0000","last_clean":"2024-04-25T20:55:10.332839+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:55:10.332839+0000","last_undegraded":"2024-04-25T20:55:10.332839+0000","last_fullsized":"2024-04-25T20:55:10.332839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.266408+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:56:09.266408+0000","last_peered":"2024-04-25T20:56:09.266408+0000","last_clean":"2024-04-25T20:56:09.266408+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:56:09.266408+0000","last_undegraded":"2024-04-25T20:56:09.266408+0000","last_fullsized":"2024-04-25T20:56:09.266408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332221+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:55:10.332221+0000","last_peered":"2024-04-25T20:55:10.332221+0000","last_clean":"2024-04-25T20:55:10.332221+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:55:10.332221+0000","last_undegraded":"2024-04-25T20:55:10.332221+0000","last_fullsized":"2024-04-25T20:55:10.332221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.261044+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:56:09.261044+0000","last_peered":"2024-04-25T20:56:09.261044+0000","last_clean":"2024-04-25T20:56:09.261044+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:56:09.261044+0000","last_undegraded":"2024-04-25T20:56:09.261044+0000","last_fullsized":"2024-04-25T20:56:09.261044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":137,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:24.502770+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:56:24.502770+0000","last_peered":"2024-04-25T20:56:24.502770+0000","last_clean":"2024-04-25T20:56:24.502770+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:56:24.502770+0000","last_undegraded":"2024-04-25T20:56:24.502770+0000","last_fullsized":"2024-04-25T20:56:24.502770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.249026+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:56:09.249026+0000","last_peered":"2024-04-25T20:56:09.249026+0000","last_clean":"2024-04-25T20:56:09.249026+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:56:09.249026+0000","last_undegraded":"2024-04-25T20:56:09.249026+0000","last_fullsized":"2024-04-25T20:56:09.249026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326638+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:55:10.326638+0000","last_peered":"2024-04-25T20:55:10.326638+0000","last_clean":"2024-04-25T20:55:10.326638+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:55:10.326638+0000","last_undegraded":"2024-04-25T20:55:10.326638+0000","last_fullsized":"2024-04-25T20:55:10.326638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256414+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:56:09.256414+0000","last_peered":"2024-04-25T20:56:09.256414+0000","last_clean":"2024-04-25T20:56:09.256414+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:56:09.256414+0000","last_undegraded":"2024-04-25T20:56:09.256414+0000","last_fullsized":"2024-04-25T20:56:09.256414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323493+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:55:10.323493+0000","last_peered":"2024-04-25T20:55:10.323493+0000","last_clean":"2024-04-25T20:55:10.323493+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:55:10.323493+0000","last_undegraded":"2024-04-25T20:55:10.323493+0000","last_fullsized":"2024-04-25T20:55:10.323493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.249750+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:56:09.249750+0000","last_peered":"2024-04-25T20:56:09.249750+0000","last_clean":"2024-04-25T20:56:09.249750+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:56:09.249750+0000","last_undegraded":"2024-04-25T20:56:09.249750+0000","last_fullsized":"2024-04-25T20:56:09.249750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324604+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:55:10.324604+0000","last_peered":"2024-04-25T20:55:10.324604+0000","last_clean":"2024-04-25T20:55:10.324604+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:55:10.324604+0000","last_undegraded":"2024-04-25T20:55:10.324604+0000","last_fullsized":"2024-04-25T20:55:10.324604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257245+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:56:09.257245+0000","last_peered":"2024-04-25T20:56:09.257245+0000","last_clean":"2024-04-25T20:56:09.257245+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:56:09.257245+0000","last_undegraded":"2024-04-25T20:56:09.257245+0000","last_fullsized":"2024-04-25T20:56:09.257245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:21.497227+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:56:21.497227+0000","last_peered":"2024-04-25T20:56:21.497227+0000","last_clean":"2024-04-25T20:56:21.497227+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:56:21.497227+0000","last_undegraded":"2024-04-25T20:56:21.497227+0000","last_fullsized":"2024-04-25T20:56:21.497227+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.266183+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:56:09.266183+0000","last_peered":"2024-04-25T20:56:09.266183+0000","last_clean":"2024-04-25T20:56:09.266183+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:56:09.266183+0000","last_undegraded":"2024-04-25T20:56:09.266183+0000","last_fullsized":"2024-04-25T20:56:09.266183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400744+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:55:10.400744+0000","last_peered":"2024-04-25T20:55:10.400744+0000","last_clean":"2024-04-25T20:55:10.400744+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:55:10.400744+0000","last_undegraded":"2024-04-25T20:55:10.400744+0000","last_fullsized":"2024-04-25T20:55:10.400744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256613+0000","last_change":"2024-04-25T20:55:10.334414+0000","last_active":"2024-04-25T20:56:09.256613+0000","last_peered":"2024-04-25T20:56:09.256613+0000","last_clean":"2024-04-25T20:56:09.256613+0000","last_became_active":"2024-04-25T20:55:10.334022+0000","last_became_peered":"2024-04-25T20:55:10.334022+0000","last_unstale":"2024-04-25T20:56:09.256613+0000","last_undegraded":"2024-04-25T20:56:09.256613+0000","last_fullsized":"2024-04-25T20:56:09.256613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:32.914876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325197+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:55:10.325197+0000","last_peered":"2024-04-25T20:55:10.325197+0000","last_clean":"2024-04-25T20:55:10.325197+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:55:10.325197+0000","last_undegraded":"2024-04-25T20:55:10.325197+0000","last_fullsized":"2024-04-25T20:55:10.325197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269705+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:56:09.269705+0000","last_peered":"2024-04-25T20:56:09.269705+0000","last_clean":"2024-04-25T20:56:09.269705+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:56:09.269705+0000","last_undegraded":"2024-04-25T20:56:09.269705+0000","last_fullsized":"2024-04-25T20:56:09.269705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":261,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:22.498607+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:56:22.498607+0000","last_peered":"2024-04-25T20:56:22.498607+0000","last_clean":"2024-04-25T20:56:22.498607+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:56:22.498607+0000","last_undegraded":"2024-04-25T20:56:22.498607+0000","last_fullsized":"2024-04-25T20:56:22.498607+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":204,"num_read_kb":161,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269164+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:56:09.269164+0000","last_peered":"2024-04-25T20:56:09.269164+0000","last_clean":"2024-04-25T20:56:09.269164+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:56:09.269164+0000","last_undegraded":"2024-04-25T20:56:09.269164+0000","last_fullsized":"2024-04-25T20:56:09.269164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331406+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:55:10.331406+0000","last_peered":"2024-04-25T20:55:10.331406+0000","last_clean":"2024-04-25T20:55:10.331406+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:55:10.331406+0000","last_undegraded":"2024-04-25T20:55:10.331406+0000","last_fullsized":"2024-04-25T20:55:10.331406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.246500+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:56:09.246500+0000","last_peered":"2024-04-25T20:56:09.246500+0000","last_clean":"2024-04-25T20:56:09.246500+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:56:09.246500+0000","last_undegraded":"2024-04-25T20:56:09.246500+0000","last_fullsized":"2024-04-25T20:56:09.246500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324988+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:55:10.324988+0000","last_peered":"2024-04-25T20:55:10.324988+0000","last_clean":"2024-04-25T20:55:10.324988+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:55:10.324988+0000","last_undegraded":"2024-04-25T20:55:10.324988+0000","last_fullsized":"2024-04-25T20:55:10.324988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268217+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:56:09.268217+0000","last_peered":"2024-04-25T20:56:09.268217+0000","last_clean":"2024-04-25T20:56:09.268217+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:56:09.268217+0000","last_undegraded":"2024-04-25T20:56:09.268217+0000","last_fullsized":"2024-04-25T20:56:09.268217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":129,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:23.500551+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:56:23.500551+0000","last_peered":"2024-04-25T20:56:23.500551+0000","last_clean":"2024-04-25T20:56:23.500551+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:56:23.500551+0000","last_undegraded":"2024-04-25T20:56:23.500551+0000","last_fullsized":"2024-04-25T20:56:23.500551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.261198+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:56:09.261198+0000","last_peered":"2024-04-25T20:56:09.261198+0000","last_clean":"2024-04-25T20:56:09.261198+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:56:09.261198+0000","last_undegraded":"2024-04-25T20:56:09.261198+0000","last_fullsized":"2024-04-25T20:56:09.261198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332109+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:55:10.332109+0000","last_peered":"2024-04-25T20:55:10.332109+0000","last_clean":"2024-04-25T20:55:10.332109+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:55:10.332109+0000","last_undegraded":"2024-04-25T20:55:10.332109+0000","last_fullsized":"2024-04-25T20:55:10.332109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.259905+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:56:09.259905+0000","last_peered":"2024-04-25T20:56:09.259905+0000","last_clean":"2024-04-25T20:56:09.259905+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:56:09.259905+0000","last_undegraded":"2024-04-25T20:56:09.259905+0000","last_fullsized":"2024-04-25T20:56:09.259905+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332534+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:55:10.332534+0000","last_peered":"2024-04-25T20:55:10.332534+0000","last_clean":"2024-04-25T20:55:10.332534+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:55:10.332534+0000","last_undegraded":"2024-04-25T20:55:10.332534+0000","last_fullsized":"2024-04-25T20:55:10.332534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256364+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:56:09.256364+0000","last_peered":"2024-04-25T20:56:09.256364+0000","last_clean":"2024-04-25T20:56:09.256364+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:56:09.256364+0000","last_undegraded":"2024-04-25T20:56:09.256364+0000","last_fullsized":"2024-04-25T20:56:09.256364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331313+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:55:10.331313+0000","last_peered":"2024-04-25T20:55:10.331313+0000","last_clean":"2024-04-25T20:55:10.331313+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:55:10.331313+0000","last_undegraded":"2024-04-25T20:55:10.331313+0000","last_fullsized":"2024-04-25T20:55:10.331313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253788+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:56:09.253788+0000","last_peered":"2024-04-25T20:56:09.253788+0000","last_clean":"2024-04-25T20:56:09.253788+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:56:09.253788+0000","last_undegraded":"2024-04-25T20:56:09.253788+0000","last_fullsized":"2024-04-25T20:56:09.253788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400439+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:55:10.400439+0000","last_peered":"2024-04-25T20:55:10.400439+0000","last_clean":"2024-04-25T20:55:10.400439+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:55:10.400439+0000","last_undegraded":"2024-04-25T20:55:10.400439+0000","last_fullsized":"2024-04-25T20:55:10.400439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255502+0000","last_change":"2024-04-25T20:55:10.332432+0000","last_active":"2024-04-25T20:56:09.255502+0000","last_peered":"2024-04-25T20:56:09.255502+0000","last_clean":"2024-04-25T20:56:09.255502+0000","last_became_active":"2024-04-25T20:55:10.332033+0000","last_became_peered":"2024-04-25T20:55:10.332033+0000","last_unstale":"2024-04-25T20:56:09.255502+0000","last_undegraded":"2024-04-25T20:56:09.255502+0000","last_fullsized":"2024-04-25T20:56:09.255502+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:34.930705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256634+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:56:09.256634+0000","last_peered":"2024-04-25T20:56:09.256634+0000","last_clean":"2024-04-25T20:56:09.256634+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:56:09.256634+0000","last_undegraded":"2024-04-25T20:56:09.256634+0000","last_fullsized":"2024-04-25T20:56:09.256634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":126,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:20.495086+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:56:20.495086+0000","last_peered":"2024-04-25T20:56:20.495086+0000","last_clean":"2024-04-25T20:56:20.495086+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:56:20.495086+0000","last_undegraded":"2024-04-25T20:56:20.495086+0000","last_fullsized":"2024-04-25T20:56:20.495086+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.270237+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:56:09.270237+0000","last_peered":"2024-04-25T20:56:09.270237+0000","last_clean":"2024-04-25T20:56:09.270237+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:56:09.270237+0000","last_undegraded":"2024-04-25T20:56:09.270237+0000","last_fullsized":"2024-04-25T20:56:09.270237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324521+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:55:10.324521+0000","last_peered":"2024-04-25T20:55:10.324521+0000","last_clean":"2024-04-25T20:55:10.324521+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:55:10.324521+0000","last_undegraded":"2024-04-25T20:55:10.324521+0000","last_fullsized":"2024-04-25T20:55:10.324521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400276+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:55:10.400276+0000","last_peered":"2024-04-25T20:55:10.400276+0000","last_clean":"2024-04-25T20:55:10.400276+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:55:10.400276+0000","last_undegraded":"2024-04-25T20:55:10.400276+0000","last_fullsized":"2024-04-25T20:55:10.400276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.248110+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:56:09.248110+0000","last_peered":"2024-04-25T20:56:09.248110+0000","last_clean":"2024-04-25T20:56:09.248110+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:56:09.248110+0000","last_undegraded":"2024-04-25T20:56:09.248110+0000","last_fullsized":"2024-04-25T20:56:09.248110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400385+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:55:10.400385+0000","last_peered":"2024-04-25T20:55:10.400385+0000","last_clean":"2024-04-25T20:55:10.400385+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:55:10.400385+0000","last_undegraded":"2024-04-25T20:55:10.400385+0000","last_fullsized":"2024-04-25T20:55:10.400385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.337866+0000","last_change":"2024-04-25T20:55:10.337450+0000","last_active":"2024-04-25T20:55:10.337866+0000","last_peered":"2024-04-25T20:55:10.337866+0000","last_clean":"2024-04-25T20:55:10.337866+0000","last_became_active":"2024-04-25T20:55:10.332821+0000","last_became_peered":"2024-04-25T20:55:10.332821+0000","last_unstale":"2024-04-25T20:55:10.337866+0000","last_undegraded":"2024-04-25T20:55:10.337866+0000","last_fullsized":"2024-04-25T20:55:10.337866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:55.070702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.239524+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:56:09.239524+0000","last_peered":"2024-04-25T20:56:09.239524+0000","last_clean":"2024-04-25T20:56:09.239524+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:56:09.239524+0000","last_undegraded":"2024-04-25T20:56:09.239524+0000","last_fullsized":"2024-04-25T20:56:09.239524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'12","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:12.405321+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:55:12.405321+0000","last_peered":"2024-04-25T20:55:12.405321+0000","last_clean":"2024-04-25T20:55:12.405321+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:55:12.405321+0000","last_undegraded":"2024-04-25T20:55:12.405321+0000","last_fullsized":"2024-04-25T20:55:12.405321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.252895+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:56:09.252895+0000","last_peered":"2024-04-25T20:56:09.252895+0000","last_clean":"2024-04-25T20:56:09.252895+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:56:09.252895+0000","last_undegraded":"2024-04-25T20:56:09.252895+0000","last_fullsized":"2024-04-25T20:56:09.252895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324679+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:55:10.324679+0000","last_peered":"2024-04-25T20:55:10.324679+0000","last_clean":"2024-04-25T20:55:10.324679+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:55:10.324679+0000","last_undegraded":"2024-04-25T20:55:10.324679+0000","last_fullsized":"2024-04-25T20:55:10.324679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255966+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:56:09.255966+0000","last_peered":"2024-04-25T20:56:09.255966+0000","last_clean":"2024-04-25T20:56:09.255966+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:56:09.255966+0000","last_undegraded":"2024-04-25T20:56:09.255966+0000","last_fullsized":"2024-04-25T20:56:09.255966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.399951+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:55:10.399951+0000","last_peered":"2024-04-25T20:55:10.399951+0000","last_clean":"2024-04-25T20:55:10.399951+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:55:10.399951+0000","last_undegraded":"2024-04-25T20:55:10.399951+0000","last_fullsized":"2024-04-25T20:55:10.399951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.221679+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:56:09.221679+0000","last_peered":"2024-04-25T20:56:09.221679+0000","last_clean":"2024-04-25T20:56:09.221679+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:56:09.221679+0000","last_undegraded":"2024-04-25T20:56:09.221679+0000","last_fullsized":"2024-04-25T20:56:09.221679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330572+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:55:10.330572+0000","last_peered":"2024-04-25T20:55:10.330572+0000","last_clean":"2024-04-25T20:55:10.330572+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:55:10.330572+0000","last_undegraded":"2024-04-25T20:55:10.330572+0000","last_fullsized":"2024-04-25T20:55:10.330572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.258085+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:56:09.258085+0000","last_peered":"2024-04-25T20:56:09.258085+0000","last_clean":"2024-04-25T20:56:09.258085+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:56:09.258085+0000","last_undegraded":"2024-04-25T20:56:09.258085+0000","last_fullsized":"2024-04-25T20:56:09.258085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.340166+0000","last_change":"2024-04-25T20:55:10.339729+0000","last_active":"2024-04-25T20:55:10.340166+0000","last_peered":"2024-04-25T20:55:10.340166+0000","last_clean":"2024-04-25T20:55:10.340166+0000","last_became_active":"2024-04-25T20:55:10.333756+0000","last_became_peered":"2024-04-25T20:55:10.333756+0000","last_unstale":"2024-04-25T20:55:10.340166+0000","last_undegraded":"2024-04-25T20:55:10.340166+0000","last_fullsized":"2024-04-25T20:55:10.340166+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:48.806227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.243564+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:56:09.243564+0000","last_peered":"2024-04-25T20:56:09.243564+0000","last_clean":"2024-04-25T20:56:09.243564+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:56:09.243564+0000","last_undegraded":"2024-04-25T20:56:09.243564+0000","last_fullsized":"2024-04-25T20:56:09.243564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331929+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:55:10.331929+0000","last_peered":"2024-04-25T20:55:10.331929+0000","last_clean":"2024-04-25T20:55:10.331929+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:55:10.331929+0000","last_undegraded":"2024-04-25T20:55:10.331929+0000","last_fullsized":"2024-04-25T20:55:10.331929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268974+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:56:09.268974+0000","last_peered":"2024-04-25T20:56:09.268974+0000","last_clean":"2024-04-25T20:56:09.268974+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:56:09.268974+0000","last_undegraded":"2024-04-25T20:56:09.268974+0000","last_fullsized":"2024-04-25T20:56:09.268974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331766+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:55:10.331766+0000","last_peered":"2024-04-25T20:55:10.331766+0000","last_clean":"2024-04-25T20:55:10.331766+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:55:10.331766+0000","last_undegraded":"2024-04-25T20:55:10.331766+0000","last_fullsized":"2024-04-25T20:55:10.331766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268340+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:56:09.268340+0000","last_peered":"2024-04-25T20:56:09.268340+0000","last_clean":"2024-04-25T20:56:09.268340+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:56:09.268340+0000","last_undegraded":"2024-04-25T20:56:09.268340+0000","last_fullsized":"2024-04-25T20:56:09.268340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325055+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:55:10.325055+0000","last_peered":"2024-04-25T20:55:10.325055+0000","last_clean":"2024-04-25T20:55:10.325055+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:55:10.325055+0000","last_undegraded":"2024-04-25T20:55:10.325055+0000","last_fullsized":"2024-04-25T20:55:10.325055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256305+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:56:09.256305+0000","last_peered":"2024-04-25T20:56:09.256305+0000","last_clean":"2024-04-25T20:56:09.256305+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:56:09.256305+0000","last_undegraded":"2024-04-25T20:56:09.256305+0000","last_fullsized":"2024-04-25T20:56:09.256305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323575+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:55:10.323575+0000","last_peered":"2024-04-25T20:55:10.323575+0000","last_clean":"2024-04-25T20:55:10.323575+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:55:10.323575+0000","last_undegraded":"2024-04-25T20:55:10.323575+0000","last_fullsized":"2024-04-25T20:55:10.323575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.264758+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:56:09.264758+0000","last_peered":"2024-04-25T20:56:09.264758+0000","last_clean":"2024-04-25T20:56:09.264758+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:56:09.264758+0000","last_undegraded":"2024-04-25T20:56:09.264758+0000","last_fullsized":"2024-04-25T20:56:09.264758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257166+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:56:09.257166+0000","last_peered":"2024-04-25T20:56:09.257166+0000","last_clean":"2024-04-25T20:56:09.257166+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:56:09.257166+0000","last_undegraded":"2024-04-25T20:56:09.257166+0000","last_fullsized":"2024-04-25T20:56:09.257166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.258929+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:56:09.258929+0000","last_peered":"2024-04-25T20:56:09.258929+0000","last_clean":"2024-04-25T20:56:09.258929+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:56:09.258929+0000","last_undegraded":"2024-04-25T20:56:09.258929+0000","last_fullsized":"2024-04-25T20:56:09.258929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253044+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:56:09.253044+0000","last_peered":"2024-04-25T20:56:09.253044+0000","last_clean":"2024-04-25T20:56:09.253044+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:56:09.253044+0000","last_undegraded":"2024-04-25T20:56:09.253044+0000","last_fullsized":"2024-04-25T20:56:09.253044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.254694+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:56:09.254694+0000","last_peered":"2024-04-25T20:56:09.254694+0000","last_clean":"2024-04-25T20:56:09.254694+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:56:09.254694+0000","last_undegraded":"2024-04-25T20:56:09.254694+0000","last_fullsized":"2024-04-25T20:56:09.254694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253665+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:56:09.253665+0000","last_peered":"2024-04-25T20:56:09.253665+0000","last_clean":"2024-04-25T20:56:09.253665+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:56:09.253665+0000","last_undegraded":"2024-04-25T20:56:09.253665+0000","last_fullsized":"2024-04-25T20:56:09.253665+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256389+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:56:09.256389+0000","last_peered":"2024-04-25T20:56:09.256389+0000","last_clean":"2024-04-25T20:56:09.256389+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:56:09.256389+0000","last_undegraded":"2024-04-25T20:56:09.256389+0000","last_fullsized":"2024-04-25T20:56:09.256389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269934+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:56:09.269934+0000","last_peered":"2024-04-25T20:56:09.269934+0000","last_clean":"2024-04-25T20:56:09.269934+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:56:09.269934+0000","last_undegraded":"2024-04-25T20:56:09.269934+0000","last_fullsized":"2024-04-25T20:56:09.269934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.254447+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:56:09.254447+0000","last_peered":"2024-04-25T20:56:09.254447+0000","last_clean":"2024-04-25T20:56:09.254447+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:56:09.254447+0000","last_undegraded":"2024-04-25T20:56:09.254447+0000","last_fullsized":"2024-04-25T20:56:09.254447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.247436+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:56:09.247436+0000","last_peered":"2024-04-25T20:56:09.247436+0000","last_clean":"2024-04-25T20:56:09.247436+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:56:09.247436+0000","last_undegraded":"2024-04-25T20:56:09.247436+0000","last_fullsized":"2024-04-25T20:56:09.247436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255396+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:56:09.255396+0000","last_peered":"2024-04-25T20:56:09.255396+0000","last_clean":"2024-04-25T20:56:09.255396+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:56:09.255396+0000","last_undegraded":"2024-04-25T20:56:09.255396+0000","last_fullsized":"2024-04-25T20:56:09.255396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.270398+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:56:09.270398+0000","last_peered":"2024-04-25T20:56:09.270398+0000","last_clean":"2024-04-25T20:56:09.270398+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:56:09.270398+0000","last_undegraded":"2024-04-25T20:56:09.270398+0000","last_fullsized":"2024-04-25T20:56:09.270398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.252498+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:56:09.252498+0000","last_peered":"2024-04-25T20:56:09.252498+0000","last_clean":"2024-04-25T20:56:09.252498+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:56:09.252498+0000","last_undegraded":"2024-04-25T20:56:09.252498+0000","last_fullsized":"2024-04-25T20:56:09.252498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.251358+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:56:09.251358+0000","last_peered":"2024-04-25T20:56:09.251358+0000","last_clean":"2024-04-25T20:56:09.251358+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:56:09.251358+0000","last_undegraded":"2024-04-25T20:56:09.251358+0000","last_fullsized":"2024-04-25T20:56:09.251358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":615,"num_read_kb":373,"num_write":171,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285609,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":495217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547246,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1147272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743474,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":495431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":495329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1087422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462978,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":494820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":555209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":496091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":557091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":495257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1087106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:56:28.330 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:56:28.681 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:29.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:29 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:29 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:29.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:29 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 -- 172.21.15.96:0/3228458524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 msgr2=0x7fb000111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 --2- 172.21.15.96:0/3228458524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb000111830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7faff800b0a0 tx=0x7faff800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 -- 172.21.15.96:0/3228458524 shutdown_connections 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 --2- 172.21.15.96:0/3228458524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb000111830 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 --2- 172.21.15.96:0/3228458524 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb000074c70 0x7fb000075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 --2- 172.21.15.96:0/3228458524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb000075590 0x7fb00010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 -- 172.21.15.96:0/3228458524 >> 172.21.15.96:0/3228458524 conn(0x7fb000079ba0 msgr2=0x7fb000079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 -- 172.21.15.96:0/3228458524 shutdown_connections 2024-04-25T20:56:30.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.316+0000 7fb0061c9700 1 -- 172.21.15.96:0/3228458524 wait complete. 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 Processor -- start 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 -- start start 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb000074c70 0x7fb00019b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb000075590 0x7fb00019b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb00019fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb000114e30 con 0x7fb00010f440 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb000114cb0 con 0x7fb000075590 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fb0061c9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb000114fb0 con 0x7fb000074c70 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb00019fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb00019fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50624/0 (socket says 172.21.15.96:50624) 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 -- 172.21.15.96:0/2299546563 learned_addr learned my addr 172.21.15.96:0/2299546563 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:30.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7fafff7fe700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb000074c70 0x7fb00019b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffeffd700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb000075590 0x7fb00019b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 -- 172.21.15.96:0/2299546563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb000074c70 msgr2=0x7fb00019b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb000074c70 0x7fb00019b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 -- 172.21.15.96:0/2299546563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb000075590 msgr2=0x7fb00019b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb000075590 0x7fb00019b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.317+0000 7faffffff700 1 -- 172.21.15.96:0/2299546563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0001a02f0 con 0x7fb00010f440 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.318+0000 7faffffff700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb00019fbb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faff8009fd0 tx=0x7faff8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.318+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff8021070 con 0x7fb00010f440 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.318+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faff8007c00 con 0x7fb00010f440 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.318+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faff80052a0 con 0x7fb00010f440 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.318+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0001a0580 con 0x7fb00010f440 2024-04-25T20:56:30.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.318+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0001a7e60 con 0x7fb00010f440 2024-04-25T20:56:30.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.319+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb000076360 con 0x7fb00010f440 2024-04-25T20:56:30.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.320+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faff8024070 con 0x7fb00010f440 2024-04-25T20:56:30.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.320+0000 7faffcff9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fafe80a3c40 0x7fafe80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:30.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.320+0000 7fafff7fe700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fafe80a3c40 0x7fafe80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:30.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.320+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7faff800f020 con 0x7fb00010f440 2024-04-25T20:56:30.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.321+0000 7fafff7fe700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fafe80a3c40 0x7fafe80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faff0005950 tx=0x7faff000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:30.324 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.324+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faff802b030 con 0x7fb00010f440 2024-04-25T20:56:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:30 smithi096 ceph-mon[29288]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:30.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.541+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb00019c640 con 0x7fafe80a3c40 2024-04-25T20:56:30.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.549+0000 7faffcff9700 1 -- 172.21.15.96:0/2299546563 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258842 (secure 0 0 0) 0x7fb00019c640 con 0x7fafe80a3c40 2024-04-25T20:56:30.553 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fafe80a3c40 msgr2=0x7fafe80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fafe80a3c40 0x7fafe80a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7faff0005950 tx=0x7faff000b410 comp rx=0 tx=0).stop 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 msgr2=0x7fb00019fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb00019fbb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7faff8009fd0 tx=0x7faff8004dc0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 shutdown_connections 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fafe80a3c40 0x7fafe80a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb00010f440 0x7fb00019fbb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb000074c70 0x7fb00019b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 --2- 172.21.15.96:0/2299546563 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb000075590 0x7fb00019b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 >> 172.21.15.96:0/2299546563 conn(0x7fb000079ba0 msgr2=0x7fb0000ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 shutdown_connections 2024-04-25T20:56:30.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:30.552+0000 7fb0061c9700 1 -- 172.21.15.96:0/2299546563 wait complete. 2024-04-25T20:56:30.556 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:56:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:30 smithi097 ceph-mon[32667]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:30 smithi139 ceph-mon[33027]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:31.247 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-04-25T20:56:29.266128+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655340,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":771,"num_read_kb":507,"num_write":354,"num_write_kb":3056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":784,"ondisk_log_size":784,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":388092,"kb_used_data":13980,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553732,"statfs":{"total":1151940427776,"available":1151543021568,"internally_reserved":0,"allocated":14315520,"data_stored":7901376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":80,"num_read_kb":45,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267515+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:56:09.267515+0000","last_peered":"2024-04-25T20:56:09.267515+0000","last_clean":"2024-04-25T20:56:09.267515+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:56:09.267515+0000","last_undegraded":"2024-04-25T20:56:09.267515+0000","last_fullsized":"2024-04-25T20:56:09.267515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256093+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:56:09.256093+0000","last_peered":"2024-04-25T20:56:09.256093+0000","last_clean":"2024-04-25T20:56:09.256093+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:56:09.256093+0000","last_undegraded":"2024-04-25T20:56:09.256093+0000","last_fullsized":"2024-04-25T20:56:09.256093+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268636+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:56:09.268636+0000","last_peered":"2024-04-25T20:56:09.268636+0000","last_clean":"2024-04-25T20:56:09.268636+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:56:09.268636+0000","last_undegraded":"2024-04-25T20:56:09.268636+0000","last_fullsized":"2024-04-25T20:56:09.268636+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257720+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:56:09.257720+0000","last_peered":"2024-04-25T20:56:09.257720+0000","last_clean":"2024-04-25T20:56:09.257720+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:56:09.257720+0000","last_undegraded":"2024-04-25T20:56:09.257720+0000","last_fullsized":"2024-04-25T20:56:09.257720+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.247668+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:56:09.247668+0000","last_peered":"2024-04-25T20:56:09.247668+0000","last_clean":"2024-04-25T20:56:09.247668+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:56:09.247668+0000","last_undegraded":"2024-04-25T20:56:09.247668+0000","last_fullsized":"2024-04-25T20:56:09.247668+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255680+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:56:09.255680+0000","last_peered":"2024-04-25T20:56:09.255680+0000","last_clean":"2024-04-25T20:56:09.255680+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:56:09.255680+0000","last_undegraded":"2024-04-25T20:56:09.255680+0000","last_fullsized":"2024-04-25T20:56:09.255680+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.245845+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:56:09.245845+0000","last_peered":"2024-04-25T20:56:09.245845+0000","last_clean":"2024-04-25T20:56:09.245845+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:56:09.245845+0000","last_undegraded":"2024-04-25T20:56:09.245845+0000","last_fullsized":"2024-04-25T20:56:09.245845+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.249193+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:56:09.249193+0000","last_peered":"2024-04-25T20:56:09.249193+0000","last_clean":"2024-04-25T20:56:09.249193+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:56:09.249193+0000","last_undegraded":"2024-04-25T20:56:09.249193+0000","last_fullsized":"2024-04-25T20:56:09.249193+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.265574+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:56:09.265574+0000","last_peered":"2024-04-25T20:56:09.265574+0000","last_clean":"2024-04-25T20:56:09.265574+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:56:09.265574+0000","last_undegraded":"2024-04-25T20:56:09.265574+0000","last_fullsized":"2024-04-25T20:56:09.265574+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257485+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:56:09.257485+0000","last_peered":"2024-04-25T20:56:09.257485+0000","last_clean":"2024-04-25T20:56:09.257485+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:56:09.257485+0000","last_undegraded":"2024-04-25T20:56:09.257485+0000","last_fullsized":"2024-04-25T20:56:09.257485+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267062+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:56:09.267062+0000","last_peered":"2024-04-25T20:56:09.267062+0000","last_clean":"2024-04-25T20:56:09.267062+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:56:09.267062+0000","last_undegraded":"2024-04-25T20:56:09.267062+0000","last_fullsized":"2024-04-25T20:56:09.267062+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255163+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:56:09.255163+0000","last_peered":"2024-04-25T20:56:09.255163+0000","last_clean":"2024-04-25T20:56:09.255163+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:56:09.255163+0000","last_undegraded":"2024-04-25T20:56:09.255163+0000","last_fullsized":"2024-04-25T20:56:09.255163+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253188+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:56:09.253188+0000","last_peered":"2024-04-25T20:56:09.253188+0000","last_clean":"2024-04-25T20:56:09.253188+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:56:09.253188+0000","last_undegraded":"2024-04-25T20:56:09.253188+0000","last_fullsized":"2024-04-25T20:56:09.253188+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256936+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:56:09.256936+0000","last_peered":"2024-04-25T20:56:09.256936+0000","last_clean":"2024-04-25T20:56:09.256936+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:56:09.256936+0000","last_undegraded":"2024-04-25T20:56:09.256936+0000","last_fullsized":"2024-04-25T20:56:09.256936+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255076+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:56:09.255076+0000","last_peered":"2024-04-25T20:56:09.255076+0000","last_clean":"2024-04-25T20:56:09.255076+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:56:09.255076+0000","last_undegraded":"2024-04-25T20:56:09.255076+0000","last_fullsized":"2024-04-25T20:56:09.255076+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267239+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:56:09.267239+0000","last_peered":"2024-04-25T20:56:09.267239+0000","last_clean":"2024-04-25T20:56:09.267239+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:56:09.267239+0000","last_undegraded":"2024-04-25T20:56:09.267239+0000","last_fullsized":"2024-04-25T20:56:09.267239+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.267728+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:56:09.267728+0000","last_peered":"2024-04-25T20:56:09.267728+0000","last_clean":"2024-04-25T20:56:09.267728+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:56:09.267728+0000","last_undegraded":"2024-04-25T20:56:09.267728+0000","last_fullsized":"2024-04-25T20:56:09.267728+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.258888+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:56:09.258888+0000","last_peered":"2024-04-25T20:56:09.258888+0000","last_clean":"2024-04-25T20:56:09.258888+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:56:09.258888+0000","last_undegraded":"2024-04-25T20:56:09.258888+0000","last_fullsized":"2024-04-25T20:56:09.258888+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.254234+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:56:09.254234+0000","last_peered":"2024-04-25T20:56:09.254234+0000","last_clean":"2024-04-25T20:56:09.254234+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:56:09.254234+0000","last_undegraded":"2024-04-25T20:56:09.254234+0000","last_fullsized":"2024-04-25T20:56:09.254234+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331587+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:55:10.331587+0000","last_peered":"2024-04-25T20:55:10.331587+0000","last_clean":"2024-04-25T20:55:10.331587+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:55:10.331587+0000","last_undegraded":"2024-04-25T20:55:10.331587+0000","last_fullsized":"2024-04-25T20:55:10.331587+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255040+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:56:09.255040+0000","last_peered":"2024-04-25T20:56:09.255040+0000","last_clean":"2024-04-25T20:56:09.255040+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:56:09.255040+0000","last_undegraded":"2024-04-25T20:56:09.255040+0000","last_fullsized":"2024-04-25T20:56:09.255040+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331113+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:55:10.331113+0000","last_peered":"2024-04-25T20:55:10.331113+0000","last_clean":"2024-04-25T20:55:10.331113+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:55:10.331113+0000","last_undegraded":"2024-04-25T20:55:10.331113+0000","last_fullsized":"2024-04-25T20:55:10.331113+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269311+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:56:09.269311+0000","last_peered":"2024-04-25T20:56:09.269311+0000","last_clean":"2024-04-25T20:56:09.269311+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:56:09.269311+0000","last_undegraded":"2024-04-25T20:56:09.269311+0000","last_fullsized":"2024-04-25T20:56:09.269311+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332839+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:55:10.332839+0000","last_peered":"2024-04-25T20:55:10.332839+0000","last_clean":"2024-04-25T20:55:10.332839+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:55:10.332839+0000","last_undegraded":"2024-04-25T20:55:10.332839+0000","last_fullsized":"2024-04-25T20:55:10.332839+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.266408+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:56:09.266408+0000","last_peered":"2024-04-25T20:56:09.266408+0000","last_clean":"2024-04-25T20:56:09.266408+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:56:09.266408+0000","last_undegraded":"2024-04-25T20:56:09.266408+0000","last_fullsized":"2024-04-25T20:56:09.266408+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332221+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:55:10.332221+0000","last_peered":"2024-04-25T20:55:10.332221+0000","last_clean":"2024-04-25T20:55:10.332221+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:55:10.332221+0000","last_undegraded":"2024-04-25T20:55:10.332221+0000","last_fullsized":"2024-04-25T20:55:10.332221+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.261044+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:56:09.261044+0000","last_peered":"2024-04-25T20:56:09.261044+0000","last_clean":"2024-04-25T20:56:09.261044+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:56:09.261044+0000","last_undegraded":"2024-04-25T20:56:09.261044+0000","last_fullsized":"2024-04-25T20:56:09.261044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"93'7","reported_seq":137,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:24.502770+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:56:24.502770+0000","last_peered":"2024-04-25T20:56:24.502770+0000","last_clean":"2024-04-25T20:56:24.502770+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:56:24.502770+0000","last_undegraded":"2024-04-25T20:56:24.502770+0000","last_fullsized":"2024-04-25T20:56:24.502770+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":104,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.249026+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:56:09.249026+0000","last_peered":"2024-04-25T20:56:09.249026+0000","last_clean":"2024-04-25T20:56:09.249026+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:56:09.249026+0000","last_undegraded":"2024-04-25T20:56:09.249026+0000","last_fullsized":"2024-04-25T20:56:09.249026+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.326638+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:55:10.326638+0000","last_peered":"2024-04-25T20:55:10.326638+0000","last_clean":"2024-04-25T20:55:10.326638+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:55:10.326638+0000","last_undegraded":"2024-04-25T20:55:10.326638+0000","last_fullsized":"2024-04-25T20:55:10.326638+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256414+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:56:09.256414+0000","last_peered":"2024-04-25T20:56:09.256414+0000","last_clean":"2024-04-25T20:56:09.256414+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:56:09.256414+0000","last_undegraded":"2024-04-25T20:56:09.256414+0000","last_fullsized":"2024-04-25T20:56:09.256414+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323493+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:55:10.323493+0000","last_peered":"2024-04-25T20:55:10.323493+0000","last_clean":"2024-04-25T20:55:10.323493+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:55:10.323493+0000","last_undegraded":"2024-04-25T20:55:10.323493+0000","last_fullsized":"2024-04-25T20:55:10.323493+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.249750+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:56:09.249750+0000","last_peered":"2024-04-25T20:56:09.249750+0000","last_clean":"2024-04-25T20:56:09.249750+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:56:09.249750+0000","last_undegraded":"2024-04-25T20:56:09.249750+0000","last_fullsized":"2024-04-25T20:56:09.249750+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324604+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:55:10.324604+0000","last_peered":"2024-04-25T20:55:10.324604+0000","last_clean":"2024-04-25T20:55:10.324604+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:55:10.324604+0000","last_undegraded":"2024-04-25T20:55:10.324604+0000","last_fullsized":"2024-04-25T20:55:10.324604+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257245+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:56:09.257245+0000","last_peered":"2024-04-25T20:56:09.257245+0000","last_clean":"2024-04-25T20:56:09.257245+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:56:09.257245+0000","last_undegraded":"2024-04-25T20:56:09.257245+0000","last_fullsized":"2024-04-25T20:56:09.257245+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'4","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:26.505158+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:56:26.505158+0000","last_peered":"2024-04-25T20:56:26.505158+0000","last_clean":"2024-04-25T20:56:26.505158+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:56:26.505158+0000","last_undegraded":"2024-04-25T20:56:26.505158+0000","last_fullsized":"2024-04-25T20:56:26.505158+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.266183+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:56:09.266183+0000","last_peered":"2024-04-25T20:56:09.266183+0000","last_clean":"2024-04-25T20:56:09.266183+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:56:09.266183+0000","last_undegraded":"2024-04-25T20:56:09.266183+0000","last_fullsized":"2024-04-25T20:56:09.266183+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400744+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:55:10.400744+0000","last_peered":"2024-04-25T20:55:10.400744+0000","last_clean":"2024-04-25T20:55:10.400744+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:55:10.400744+0000","last_undegraded":"2024-04-25T20:55:10.400744+0000","last_fullsized":"2024-04-25T20:55:10.400744+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256613+0000","last_change":"2024-04-25T20:55:10.334414+0000","last_active":"2024-04-25T20:56:09.256613+0000","last_peered":"2024-04-25T20:56:09.256613+0000","last_clean":"2024-04-25T20:56:09.256613+0000","last_became_active":"2024-04-25T20:55:10.334022+0000","last_became_peered":"2024-04-25T20:55:10.334022+0000","last_unstale":"2024-04-25T20:56:09.256613+0000","last_undegraded":"2024-04-25T20:56:09.256613+0000","last_fullsized":"2024-04-25T20:56:09.256613+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:32.914876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325197+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:55:10.325197+0000","last_peered":"2024-04-25T20:55:10.325197+0000","last_clean":"2024-04-25T20:55:10.325197+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:55:10.325197+0000","last_undegraded":"2024-04-25T20:55:10.325197+0000","last_fullsized":"2024-04-25T20:55:10.325197+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269705+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:56:09.269705+0000","last_peered":"2024-04-25T20:56:09.269705+0000","last_clean":"2024-04-25T20:56:09.269705+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:56:09.269705+0000","last_undegraded":"2024-04-25T20:56:09.269705+0000","last_fullsized":"2024-04-25T20:56:09.269705+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"93'32","reported_seq":271,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:27.506639+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:56:27.506639+0000","last_peered":"2024-04-25T20:56:27.506639+0000","last_clean":"2024-04-25T20:56:27.506639+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:56:27.506639+0000","last_undegraded":"2024-04-25T20:56:27.506639+0000","last_fullsized":"2024-04-25T20:56:27.506639+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60596,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":214,"num_read_kb":166,"num_write":33,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269164+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:56:09.269164+0000","last_peered":"2024-04-25T20:56:09.269164+0000","last_clean":"2024-04-25T20:56:09.269164+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:56:09.269164+0000","last_undegraded":"2024-04-25T20:56:09.269164+0000","last_fullsized":"2024-04-25T20:56:09.269164+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331406+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:55:10.331406+0000","last_peered":"2024-04-25T20:55:10.331406+0000","last_clean":"2024-04-25T20:55:10.331406+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:55:10.331406+0000","last_undegraded":"2024-04-25T20:55:10.331406+0000","last_fullsized":"2024-04-25T20:55:10.331406+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.246500+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:56:09.246500+0000","last_peered":"2024-04-25T20:56:09.246500+0000","last_clean":"2024-04-25T20:56:09.246500+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:56:09.246500+0000","last_undegraded":"2024-04-25T20:56:09.246500+0000","last_fullsized":"2024-04-25T20:56:09.246500+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324988+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:55:10.324988+0000","last_peered":"2024-04-25T20:55:10.324988+0000","last_clean":"2024-04-25T20:55:10.324988+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:55:10.324988+0000","last_undegraded":"2024-04-25T20:55:10.324988+0000","last_fullsized":"2024-04-25T20:55:10.324988+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268217+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:56:09.268217+0000","last_peered":"2024-04-25T20:56:09.268217+0000","last_clean":"2024-04-25T20:56:09.268217+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:56:09.268217+0000","last_undegraded":"2024-04-25T20:56:09.268217+0000","last_fullsized":"2024-04-25T20:56:09.268217+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":129,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:23.500551+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:56:23.500551+0000","last_peered":"2024-04-25T20:56:23.500551+0000","last_clean":"2024-04-25T20:56:23.500551+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:56:23.500551+0000","last_undegraded":"2024-04-25T20:56:23.500551+0000","last_fullsized":"2024-04-25T20:56:23.500551+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.261198+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:56:09.261198+0000","last_peered":"2024-04-25T20:56:09.261198+0000","last_clean":"2024-04-25T20:56:09.261198+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:56:09.261198+0000","last_undegraded":"2024-04-25T20:56:09.261198+0000","last_fullsized":"2024-04-25T20:56:09.261198+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":42,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332109+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:55:10.332109+0000","last_peered":"2024-04-25T20:55:10.332109+0000","last_clean":"2024-04-25T20:55:10.332109+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:55:10.332109+0000","last_undegraded":"2024-04-25T20:55:10.332109+0000","last_fullsized":"2024-04-25T20:55:10.332109+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.259905+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:56:09.259905+0000","last_peered":"2024-04-25T20:56:09.259905+0000","last_clean":"2024-04-25T20:56:09.259905+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:56:09.259905+0000","last_undegraded":"2024-04-25T20:56:09.259905+0000","last_fullsized":"2024-04-25T20:56:09.259905+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.332534+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:55:10.332534+0000","last_peered":"2024-04-25T20:55:10.332534+0000","last_clean":"2024-04-25T20:55:10.332534+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:55:10.332534+0000","last_undegraded":"2024-04-25T20:55:10.332534+0000","last_fullsized":"2024-04-25T20:55:10.332534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256364+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:56:09.256364+0000","last_peered":"2024-04-25T20:56:09.256364+0000","last_clean":"2024-04-25T20:56:09.256364+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:56:09.256364+0000","last_undegraded":"2024-04-25T20:56:09.256364+0000","last_fullsized":"2024-04-25T20:56:09.256364+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331313+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:55:10.331313+0000","last_peered":"2024-04-25T20:55:10.331313+0000","last_clean":"2024-04-25T20:55:10.331313+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:55:10.331313+0000","last_undegraded":"2024-04-25T20:55:10.331313+0000","last_fullsized":"2024-04-25T20:55:10.331313+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253788+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:56:09.253788+0000","last_peered":"2024-04-25T20:56:09.253788+0000","last_clean":"2024-04-25T20:56:09.253788+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:56:09.253788+0000","last_undegraded":"2024-04-25T20:56:09.253788+0000","last_fullsized":"2024-04-25T20:56:09.253788+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400439+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:55:10.400439+0000","last_peered":"2024-04-25T20:55:10.400439+0000","last_clean":"2024-04-25T20:55:10.400439+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:55:10.400439+0000","last_undegraded":"2024-04-25T20:55:10.400439+0000","last_fullsized":"2024-04-25T20:55:10.400439+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255502+0000","last_change":"2024-04-25T20:55:10.332432+0000","last_active":"2024-04-25T20:56:09.255502+0000","last_peered":"2024-04-25T20:56:09.255502+0000","last_clean":"2024-04-25T20:56:09.255502+0000","last_became_active":"2024-04-25T20:55:10.332033+0000","last_became_peered":"2024-04-25T20:55:10.332033+0000","last_unstale":"2024-04-25T20:56:09.255502+0000","last_undegraded":"2024-04-25T20:56:09.255502+0000","last_fullsized":"2024-04-25T20:56:09.255502+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:34.930705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256634+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:56:09.256634+0000","last_peered":"2024-04-25T20:56:09.256634+0000","last_clean":"2024-04-25T20:56:09.256634+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:56:09.256634+0000","last_undegraded":"2024-04-25T20:56:09.256634+0000","last_fullsized":"2024-04-25T20:56:09.256634+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":131,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:25.502735+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:56:25.502735+0000","last_peered":"2024-04-25T20:56:25.502735+0000","last_clean":"2024-04-25T20:56:25.502735+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:56:25.502735+0000","last_undegraded":"2024-04-25T20:56:25.502735+0000","last_fullsized":"2024-04-25T20:56:25.502735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.270237+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:56:09.270237+0000","last_peered":"2024-04-25T20:56:09.270237+0000","last_clean":"2024-04-25T20:56:09.270237+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:56:09.270237+0000","last_undegraded":"2024-04-25T20:56:09.270237+0000","last_fullsized":"2024-04-25T20:56:09.270237+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324521+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:55:10.324521+0000","last_peered":"2024-04-25T20:55:10.324521+0000","last_clean":"2024-04-25T20:55:10.324521+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:55:10.324521+0000","last_undegraded":"2024-04-25T20:55:10.324521+0000","last_fullsized":"2024-04-25T20:55:10.324521+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":362,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400276+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:55:10.400276+0000","last_peered":"2024-04-25T20:55:10.400276+0000","last_clean":"2024-04-25T20:55:10.400276+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:55:10.400276+0000","last_undegraded":"2024-04-25T20:55:10.400276+0000","last_fullsized":"2024-04-25T20:55:10.400276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.248110+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:56:09.248110+0000","last_peered":"2024-04-25T20:56:09.248110+0000","last_clean":"2024-04-25T20:56:09.248110+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:56:09.248110+0000","last_undegraded":"2024-04-25T20:56:09.248110+0000","last_fullsized":"2024-04-25T20:56:09.248110+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.400385+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:55:10.400385+0000","last_peered":"2024-04-25T20:55:10.400385+0000","last_clean":"2024-04-25T20:55:10.400385+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:55:10.400385+0000","last_undegraded":"2024-04-25T20:55:10.400385+0000","last_fullsized":"2024-04-25T20:55:10.400385+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.337866+0000","last_change":"2024-04-25T20:55:10.337450+0000","last_active":"2024-04-25T20:55:10.337866+0000","last_peered":"2024-04-25T20:55:10.337866+0000","last_clean":"2024-04-25T20:55:10.337866+0000","last_became_active":"2024-04-25T20:55:10.332821+0000","last_became_peered":"2024-04-25T20:55:10.332821+0000","last_unstale":"2024-04-25T20:55:10.337866+0000","last_undegraded":"2024-04-25T20:55:10.337866+0000","last_fullsized":"2024-04-25T20:55:10.337866+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:55.070702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.239524+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:56:09.239524+0000","last_peered":"2024-04-25T20:56:09.239524+0000","last_clean":"2024-04-25T20:56:09.239524+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:56:09.239524+0000","last_undegraded":"2024-04-25T20:56:09.239524+0000","last_fullsized":"2024-04-25T20:56:09.239524+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"93'12","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:12.405321+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:55:12.405321+0000","last_peered":"2024-04-25T20:55:12.405321+0000","last_clean":"2024-04-25T20:55:12.405321+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:55:12.405321+0000","last_undegraded":"2024-04-25T20:55:12.405321+0000","last_fullsized":"2024-04-25T20:55:12.405321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.252895+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:56:09.252895+0000","last_peered":"2024-04-25T20:56:09.252895+0000","last_clean":"2024-04-25T20:56:09.252895+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:56:09.252895+0000","last_undegraded":"2024-04-25T20:56:09.252895+0000","last_fullsized":"2024-04-25T20:56:09.252895+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.324679+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:55:10.324679+0000","last_peered":"2024-04-25T20:55:10.324679+0000","last_clean":"2024-04-25T20:55:10.324679+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:55:10.324679+0000","last_undegraded":"2024-04-25T20:55:10.324679+0000","last_fullsized":"2024-04-25T20:55:10.324679+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255966+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:56:09.255966+0000","last_peered":"2024-04-25T20:56:09.255966+0000","last_clean":"2024-04-25T20:56:09.255966+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:56:09.255966+0000","last_undegraded":"2024-04-25T20:56:09.255966+0000","last_fullsized":"2024-04-25T20:56:09.255966+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.399951+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:55:10.399951+0000","last_peered":"2024-04-25T20:55:10.399951+0000","last_clean":"2024-04-25T20:55:10.399951+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:55:10.399951+0000","last_undegraded":"2024-04-25T20:55:10.399951+0000","last_fullsized":"2024-04-25T20:55:10.399951+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":22,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.221679+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:56:09.221679+0000","last_peered":"2024-04-25T20:56:09.221679+0000","last_clean":"2024-04-25T20:56:09.221679+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:56:09.221679+0000","last_undegraded":"2024-04-25T20:56:09.221679+0000","last_fullsized":"2024-04-25T20:56:09.221679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.330572+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:55:10.330572+0000","last_peered":"2024-04-25T20:55:10.330572+0000","last_clean":"2024-04-25T20:55:10.330572+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:55:10.330572+0000","last_undegraded":"2024-04-25T20:55:10.330572+0000","last_fullsized":"2024-04-25T20:55:10.330572+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.258085+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:56:09.258085+0000","last_peered":"2024-04-25T20:56:09.258085+0000","last_clean":"2024-04-25T20:56:09.258085+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:56:09.258085+0000","last_undegraded":"2024-04-25T20:56:09.258085+0000","last_fullsized":"2024-04-25T20:56:09.258085+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"91'1","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.340166+0000","last_change":"2024-04-25T20:55:10.339729+0000","last_active":"2024-04-25T20:55:10.340166+0000","last_peered":"2024-04-25T20:55:10.340166+0000","last_clean":"2024-04-25T20:55:10.340166+0000","last_became_active":"2024-04-25T20:55:10.333756+0000","last_became_peered":"2024-04-25T20:55:10.333756+0000","last_unstale":"2024-04-25T20:55:10.340166+0000","last_undegraded":"2024-04-25T20:55:10.340166+0000","last_fullsized":"2024-04-25T20:55:10.340166+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:48.806227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.243564+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:56:09.243564+0000","last_peered":"2024-04-25T20:56:09.243564+0000","last_clean":"2024-04-25T20:56:09.243564+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:56:09.243564+0000","last_undegraded":"2024-04-25T20:56:09.243564+0000","last_fullsized":"2024-04-25T20:56:09.243564+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331929+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:55:10.331929+0000","last_peered":"2024-04-25T20:55:10.331929+0000","last_clean":"2024-04-25T20:55:10.331929+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:55:10.331929+0000","last_undegraded":"2024-04-25T20:55:10.331929+0000","last_fullsized":"2024-04-25T20:55:10.331929+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268974+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:56:09.268974+0000","last_peered":"2024-04-25T20:56:09.268974+0000","last_clean":"2024-04-25T20:56:09.268974+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:56:09.268974+0000","last_undegraded":"2024-04-25T20:56:09.268974+0000","last_fullsized":"2024-04-25T20:56:09.268974+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.331766+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:55:10.331766+0000","last_peered":"2024-04-25T20:55:10.331766+0000","last_clean":"2024-04-25T20:55:10.331766+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:55:10.331766+0000","last_undegraded":"2024-04-25T20:55:10.331766+0000","last_fullsized":"2024-04-25T20:55:10.331766+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.268340+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:56:09.268340+0000","last_peered":"2024-04-25T20:56:09.268340+0000","last_clean":"2024-04-25T20:56:09.268340+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:56:09.268340+0000","last_undegraded":"2024-04-25T20:56:09.268340+0000","last_fullsized":"2024-04-25T20:56:09.268340+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"91'1","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.325055+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:55:10.325055+0000","last_peered":"2024-04-25T20:55:10.325055+0000","last_clean":"2024-04-25T20:55:10.325055+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:55:10.325055+0000","last_undegraded":"2024-04-25T20:55:10.325055+0000","last_fullsized":"2024-04-25T20:55:10.325055+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256305+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:56:09.256305+0000","last_peered":"2024-04-25T20:56:09.256305+0000","last_clean":"2024-04-25T20:56:09.256305+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:56:09.256305+0000","last_undegraded":"2024-04-25T20:56:09.256305+0000","last_fullsized":"2024-04-25T20:56:09.256305+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:55:10.323575+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:55:10.323575+0000","last_peered":"2024-04-25T20:55:10.323575+0000","last_clean":"2024-04-25T20:55:10.323575+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:55:10.323575+0000","last_undegraded":"2024-04-25T20:55:10.323575+0000","last_fullsized":"2024-04-25T20:55:10.323575+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.264758+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:56:09.264758+0000","last_peered":"2024-04-25T20:56:09.264758+0000","last_clean":"2024-04-25T20:56:09.264758+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:56:09.264758+0000","last_undegraded":"2024-04-25T20:56:09.264758+0000","last_fullsized":"2024-04-25T20:56:09.264758+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.257166+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:56:09.257166+0000","last_peered":"2024-04-25T20:56:09.257166+0000","last_clean":"2024-04-25T20:56:09.257166+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:56:09.257166+0000","last_undegraded":"2024-04-25T20:56:09.257166+0000","last_fullsized":"2024-04-25T20:56:09.257166+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.258929+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:56:09.258929+0000","last_peered":"2024-04-25T20:56:09.258929+0000","last_clean":"2024-04-25T20:56:09.258929+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:56:09.258929+0000","last_undegraded":"2024-04-25T20:56:09.258929+0000","last_fullsized":"2024-04-25T20:56:09.258929+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253044+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:56:09.253044+0000","last_peered":"2024-04-25T20:56:09.253044+0000","last_clean":"2024-04-25T20:56:09.253044+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:56:09.253044+0000","last_undegraded":"2024-04-25T20:56:09.253044+0000","last_fullsized":"2024-04-25T20:56:09.253044+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":25,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.254694+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:56:09.254694+0000","last_peered":"2024-04-25T20:56:09.254694+0000","last_clean":"2024-04-25T20:56:09.254694+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:56:09.254694+0000","last_undegraded":"2024-04-25T20:56:09.254694+0000","last_fullsized":"2024-04-25T20:56:09.254694+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.253665+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:56:09.253665+0000","last_peered":"2024-04-25T20:56:09.253665+0000","last_clean":"2024-04-25T20:56:09.253665+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:56:09.253665+0000","last_undegraded":"2024-04-25T20:56:09.253665+0000","last_fullsized":"2024-04-25T20:56:09.253665+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":24,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.256389+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:56:09.256389+0000","last_peered":"2024-04-25T20:56:09.256389+0000","last_clean":"2024-04-25T20:56:09.256389+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:56:09.256389+0000","last_undegraded":"2024-04-25T20:56:09.256389+0000","last_fullsized":"2024-04-25T20:56:09.256389+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.269934+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:56:09.269934+0000","last_peered":"2024-04-25T20:56:09.269934+0000","last_clean":"2024-04-25T20:56:09.269934+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:56:09.269934+0000","last_undegraded":"2024-04-25T20:56:09.269934+0000","last_fullsized":"2024-04-25T20:56:09.269934+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":20,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.254447+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:56:09.254447+0000","last_peered":"2024-04-25T20:56:09.254447+0000","last_clean":"2024-04-25T20:56:09.254447+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:56:09.254447+0000","last_undegraded":"2024-04-25T20:56:09.254447+0000","last_fullsized":"2024-04-25T20:56:09.254447+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.247436+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:56:09.247436+0000","last_peered":"2024-04-25T20:56:09.247436+0000","last_clean":"2024-04-25T20:56:09.247436+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:56:09.247436+0000","last_undegraded":"2024-04-25T20:56:09.247436+0000","last_fullsized":"2024-04-25T20:56:09.247436+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.255396+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:56:09.255396+0000","last_peered":"2024-04-25T20:56:09.255396+0000","last_clean":"2024-04-25T20:56:09.255396+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:56:09.255396+0000","last_undegraded":"2024-04-25T20:56:09.255396+0000","last_fullsized":"2024-04-25T20:56:09.255396+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.270398+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:56:09.270398+0000","last_peered":"2024-04-25T20:56:09.270398+0000","last_clean":"2024-04-25T20:56:09.270398+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:56:09.270398+0000","last_undegraded":"2024-04-25T20:56:09.270398+0000","last_fullsized":"2024-04-25T20:56:09.270398+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.252498+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:56:09.252498+0000","last_peered":"2024-04-25T20:56:09.252498+0000","last_clean":"2024-04-25T20:56:09.252498+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:56:09.252498+0000","last_undegraded":"2024-04-25T20:56:09.252498+0000","last_fullsized":"2024-04-25T20:56:09.252498+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-25T20:56:09.251358+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:56:09.251358+0000","last_peered":"2024-04-25T20:56:09.251358+0000","last_clean":"2024-04-25T20:56:09.251358+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:56:09.251358+0000","last_undegraded":"2024-04-25T20:56:09.251358+0000","last_fullsized":"2024-04-25T20:56:09.251358+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64738,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":635,"num_read_kb":388,"num_write":171,"num_write_kb":116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285610,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":495217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547247,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37004,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708148,"statfs":{"total":95995035648,"available":95957143552,"internally_reserved":0,"allocated":1708032,"data_stored":1147272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743475,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36340,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708812,"statfs":{"total":95995035648,"available":95957823488,"internally_reserved":0,"allocated":1028096,"data_stored":495431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005111,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1024000,"data_stored":495329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1087422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462979,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":494820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1077248,"data_stored":555209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1036288,"data_stored":496091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411414,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":557091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149776,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":495131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888139,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1019904,"data_stored":495257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28612,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716540,"statfs":{"total":95995035648,"available":95965736960,"internally_reserved":0,"allocated":1634304,"data_stored":1087106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:56:31.251 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T20:56:31.251 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T20:56:31.251 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-25T20:56:31.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-25T20:56:31.508 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:32 smithi097 ceph-mon[32667]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:56:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:32 smithi097 ceph-mon[32667]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:32.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:32 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:32 smithi139 ceph-mon[33027]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:56:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:32 smithi139 ceph-mon[33027]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:32 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:32.641 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:32 smithi096 ceph-mon[29288]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:56:32.642 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:32 smithi096 ceph-mon[29288]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:32.642 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:32.642 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:32 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:33.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.168+0000 7f4584b14700 1 -- 172.21.15.96:0/12849595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458010f440 msgr2=0x7f4580111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.168+0000 7f4584b14700 1 --2- 172.21.15.96:0/12849595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458010f440 0x7f4580111830 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f457800b0a0 tx=0x7f457800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 -- 172.21.15.96:0/12849595 shutdown_connections 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 --2- 172.21.15.96:0/12849595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458010f440 0x7f4580111830 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 --2- 172.21.15.96:0/12849595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580074c70 0x7f4580075050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 --2- 172.21.15.96:0/12849595 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4580075590 0x7f458010ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 -- 172.21.15.96:0/12849595 >> 172.21.15.96:0/12849595 conn(0x7f4580079ba0 msgr2=0x7f4580079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 -- 172.21.15.96:0/12849595 shutdown_connections 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 -- 172.21.15.96:0/12849595 wait complete. 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 Processor -- start 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.169+0000 7f4584b14700 1 -- start start 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 0x7f458019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580075590 0x7f458019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458010f440 0x7f458019fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:33.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4580114ec0 con 0x7f4580074c70 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4580114d40 con 0x7f4580075590 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4580115040 con 0x7f458010f440 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 0x7f458019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 0x7f458019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50652/0 (socket says 172.21.15.96:50652) 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 -- 172.21.15.96:0/4085250170 learned_addr learned my addr 172.21.15.96:0/4085250170 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457ed9d700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458010f440 0x7f458019fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457dd9b700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580075590 0x7f458019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 -- 172.21.15.96:0/4085250170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458010f440 msgr2=0x7f458019fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458010f440 0x7f458019fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 -- 172.21.15.96:0/4085250170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580075590 msgr2=0x7f458019b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580075590 0x7f458019b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 -- 172.21.15.96:0/4085250170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45801a04c0 con 0x7f4580074c70 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f457e59c700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 0x7f458019b4b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f457000b530 tx=0x7f457000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570015020 con 0x7f4580074c70 2024-04-25T20:56:33.173 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4570011ce0 con 0x7f4580074c70 2024-04-25T20:56:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45801a07b0 con 0x7f4580074c70 2024-04-25T20:56:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4570010e40 con 0x7f4580074c70 2024-04-25T20:56:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.170+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45801a8030 con 0x7f4580074c70 2024-04-25T20:56:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.171+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4580069730 con 0x7f4580074c70 2024-04-25T20:56:33.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.173+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4570018020 con 0x7f4580074c70 2024-04-25T20:56:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.173+0000 7f456f7fe700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f45680a3c40 0x7f45680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.173+0000 7f457dd9b700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f45680a3c40 0x7f45680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.173+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f45700d0130 con 0x7f4580074c70 2024-04-25T20:56:33.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.174+0000 7f457dd9b700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f45680a3c40 0x7f45680a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4574006fd0 tx=0x7f4574008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:33.177 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.177+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f457001f020 con 0x7f4580074c70 2024-04-25T20:56:33.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.427+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f458019c810 con 0x7f4580074c70 2024-04-25T20:56:33.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.427+0000 7f456f7fe700 1 -- 172.21.15.96:0/4085250170 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f457009cbb0 con 0x7f4580074c70 2024-04-25T20:56:33.428 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:33.428 INFO:teuthology.orchestra.run.smithi096.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f45680a3c40 msgr2=0x7f45680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f45680a3c40 0x7f45680a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4574006fd0 tx=0x7f4574008040 comp rx=0 tx=0).stop 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 msgr2=0x7f458019b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 0x7f458019b4b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f457000b530 tx=0x7f457000b840 comp rx=0 tx=0).stop 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 shutdown_connections 2024-04-25T20:56:33.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f45680a3c40 0x7f45680a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f458010f440 0x7f458019fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4580074c70 0x7f458019b4b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 --2- 172.21.15.96:0/4085250170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4580075590 0x7f458019b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 >> 172.21.15.96:0/4085250170 conn(0x7f4580079ba0 msgr2=0x7f4580101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 shutdown_connections 2024-04-25T20:56:33.431 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:33.429+0000 7f4584b14700 1 -- 172.21.15.96:0/4085250170 wait complete. 2024-04-25T20:56:33.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:33.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:33.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:33 smithi097 ceph-mon[32667]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:33.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:33.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:33.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:33 smithi139 ceph-mon[33027]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:33.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:56:33.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:56:33.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:33 smithi096 ceph-mon[29288]: from='mgr.14152 172.21.15.96:0/653631843' entity='mgr.x' 2024-04-25T20:56:34.108 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-25T20:56:34.109 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:34.347 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:34 smithi097 ceph-mon[32667]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:34 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4085250170' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T20:56:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:34 smithi139 ceph-mon[33027]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:34 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4085250170' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T20:56:34.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:34 smithi096 ceph-mon[29288]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:34.635 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:34 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4085250170' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-25T20:56:36.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 -- 172.21.15.96:0/4188861310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c107920 msgr2=0x7ff48c107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.050 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 --2- 172.21.15.96:0/4188861310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c107920 0x7ff48c107d00 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7ff480009990 tx=0x7ff480009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 -- 172.21.15.96:0/4188861310 shutdown_connections 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 --2- 172.21.15.96:0/4188861310 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c102330 0x7ff48c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 --2- 172.21.15.96:0/4188861310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c107920 0x7ff48c107d00 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 --2- 172.21.15.96:0/4188861310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c101970 0x7ff48c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 -- 172.21.15.96:0/4188861310 >> 172.21.15.96:0/4188861310 conn(0x7ff48c078910 msgr2=0x7ff48c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 -- 172.21.15.96:0/4188861310 shutdown_connections 2024-04-25T20:56:36.051 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.049+0000 7ff491231700 1 -- 172.21.15.96:0/4188861310 wait complete. 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 Processor -- start 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 -- start start 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 0x7ff48c199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c102330 0x7ff48c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c107920 0x7ff48c19dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c111b60 con 0x7ff48c101970 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c1119e0 con 0x7ff48c102330 2024-04-25T20:56:36.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff491231700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff48c111ce0 con 0x7ff48c107920 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff48ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 0x7ff48c199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff48ad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 0x7ff48c199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50676/0 (socket says 172.21.15.96:50676) 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.050+0000 7ff48ad9d700 1 -- 172.21.15.96:0/3234202711 learned_addr learned my addr 172.21.15.96:0/3234202711 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48a59c700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c102330 0x7ff48c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48b59e700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c107920 0x7ff48c19dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48ad9d700 1 -- 172.21.15.96:0/3234202711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c107920 msgr2=0x7ff48c19dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48ad9d700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c107920 0x7ff48c19dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48ad9d700 1 -- 172.21.15.96:0/3234202711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c102330 msgr2=0x7ff48c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48ad9d700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c102330 0x7ff48c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48ad9d700 1 -- 172.21.15.96:0/3234202711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff48c19e220 con 0x7ff48c101970 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48a59c700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c102330 0x7ff48c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff48ad9d700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 0x7ff48c199260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff4800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:36.054 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff480027070 con 0x7ff48c101970 2024-04-25T20:56:36.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4800041d0 con 0x7ff48c101970 2024-04-25T20:56:36.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff48c19e4b0 con 0x7ff48c101970 2024-04-25T20:56:36.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff48000f460 con 0x7ff48c101970 2024-04-25T20:56:36.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.051+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff48c1a5d90 con 0x7ff48c101970 2024-04-25T20:56:36.055 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.052+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff48c069730 con 0x7ff48c101970 2024-04-25T20:56:36.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.053+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff48002a070 con 0x7ff48c101970 2024-04-25T20:56:36.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.054+0000 7ff473fff700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff4740a3c40 0x7ff4740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:36.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.054+0000 7ff48a59c700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff4740a3c40 0x7ff4740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:36.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.054+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7ff4800d1740 con 0x7ff48c101970 2024-04-25T20:56:36.056 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.054+0000 7ff48a59c700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff4740a3c40 0x7ff4740a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff48c079110 tx=0x7ff47c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:36.058 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.057+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff480031070 con 0x7ff48c101970 2024-04-25T20:56:36.301 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.300+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff48c02df80 con 0x7ff48c101970 2024-04-25T20:56:36.302 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.301+0000 7ff473fff700 1 -- 172.21.15.96:0/3234202711 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7ff4800a4030 con 0x7ff48c101970 2024-04-25T20:56:36.302 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:36.302 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:54:49.357397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff4740a3c40 msgr2=0x7ff4740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff4740a3c40 0x7ff4740a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff48c079110 tx=0x7ff47c008040 comp rx=0 tx=0).stop 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 msgr2=0x7ff48c199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 0x7ff48c199260 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff480000c00 tx=0x7ff4800047b0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 shutdown_connections 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7ff4740a3c40 0x7ff4740a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.304 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff48c107920 0x7ff48c19dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff48c101970 0x7ff48c199260 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 --2- 172.21.15.96:0/3234202711 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff48c102330 0x7ff48c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 >> 172.21.15.96:0/3234202711 conn(0x7ff48c078910 msgr2=0x7ff48c06b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 shutdown_connections 2024-04-25T20:56:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:36.303+0000 7ff491231700 1 -- 172.21.15.96:0/3234202711 wait complete. 2024-04-25T20:56:36.305 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:56:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:36 smithi097 ceph-mon[32667]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:36 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3234202711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:36 smithi139 ceph-mon[33027]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:36 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3234202711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:36.653 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:36 smithi096 ceph-mon[29288]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:36.653 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:36 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3234202711' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:36.991 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:37.228 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:38 smithi097 ceph-mon[32667]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:38 smithi139 ceph-mon[33027]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:38.644 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:38 smithi096 ceph-mon[29288]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:38.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 -- 172.21.15.96:0/4160661951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 msgr2=0x7fb06c102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:38.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 --2- 172.21.15.96:0/4160661951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c102f10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb060009990 tx=0x7fb060009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 -- 172.21.15.96:0/4160661951 shutdown_connections 2024-04-25T20:56:38.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 --2- 172.21.15.96:0/4160661951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c103450 0x7fb06c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 --2- 172.21.15.96:0/4160661951 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 0x7fb06c108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 --2- 172.21.15.96:0/4160661951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c102f10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 -- 172.21.15.96:0/4160661951 >> 172.21.15.96:0/4160661951 conn(0x7fb06c079ba0 msgr2=0x7fb06c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 -- 172.21.15.96:0/4160661951 shutdown_connections 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.873+0000 7fb070811700 1 -- 172.21.15.96:0/4160661951 wait complete. 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 Processor -- start 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 -- start start 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c1992e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c103450 0x7fb06c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 0x7fb06c19dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112ce0 con 0x7fb06c102a90 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112b60 con 0x7fb06c103450 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb070811700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb06c112e60 con 0x7fb06c108a50 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb06a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c1992e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb06a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c1992e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55696/0 (socket says 172.21.15.96:55696) 2024-04-25T20:56:38.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb06a59c700 1 -- 172.21.15.96:0/4166731171 learned_addr learned my addr 172.21.15.96:0/4166731171 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.874+0000 7fb069d9b700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c103450 0x7fb06c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06ad9d700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 0x7fb06c19dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06a59c700 1 -- 172.21.15.96:0/4166731171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 msgr2=0x7fb06c19dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06a59c700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 0x7fb06c19dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06a59c700 1 -- 172.21.15.96:0/4166731171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c103450 msgr2=0x7fb06c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06a59c700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c103450 0x7fb06c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06a59c700 1 -- 172.21.15.96:0/4166731171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb06c19e2f0 con 0x7fb06c102a90 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06ad9d700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 0x7fb06c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb06a59c700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c1992e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ba50 tx=0x7fb06400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb064019070 con 0x7fb06c102a90 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb06c19e5e0 con 0x7fb06c102a90 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb06c1a5e60 con 0x7fb06c102a90 2024-04-25T20:56:38.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb064014440 con 0x7fb06c102a90 2024-04-25T20:56:38.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.875+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb06401d610 con 0x7fb06c102a90 2024-04-25T20:56:38.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.876+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb04c005330 con 0x7fb06c102a90 2024-04-25T20:56:38.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.877+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb06401c070 con 0x7fb06c102a90 2024-04-25T20:56:38.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.878+0000 7fb05b7fe700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0540a3c40 0x7fb0540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:38.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.878+0000 7fb069d9b700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0540a3c40 0x7fb0540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:38.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.878+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fb0640cf6f0 con 0x7fb06c102a90 2024-04-25T20:56:38.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.878+0000 7fb069d9b700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0540a3c40 0x7fb0540a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb060006010 tx=0x7fb06001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:38.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:38.882+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb06409c170 con 0x7fb06c102a90 2024-04-25T20:56:39.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.125+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb04c0058f0 con 0x7fb06c102a90 2024-04-25T20:56:39.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.126+0000 7fb05b7fe700 1 -- 172.21.15.96:0/4166731171 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7fb06409ba70 con 0x7fb06c102a90 2024-04-25T20:56:39.127 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:39.127 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:54:49.357397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:56:39.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0540a3c40 msgr2=0x7fb0540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0540a3c40 0x7fb0540a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb060006010 tx=0x7fb06001a040 comp rx=0 tx=0).stop 2024-04-25T20:56:39.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 msgr2=0x7fb06c1992e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:39.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c1992e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb06400ba50 tx=0x7fb06400be10 comp rx=0 tx=0).stop 2024-04-25T20:56:39.129 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 shutdown_connections 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0540a3c40 0x7fb0540a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb06c108a50 0x7fb06c19dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb06c102a90 0x7fb06c1992e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 --2- 172.21.15.96:0/4166731171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb06c103450 0x7fb06c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 >> 172.21.15.96:0/4166731171 conn(0x7fb06c079ba0 msgr2=0x7fb06c1019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 shutdown_connections 2024-04-25T20:56:39.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:39.128+0000 7fb070811700 1 -- 172.21.15.96:0/4166731171 wait complete. 2024-04-25T20:56:39.131 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:56:39.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:39 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4166731171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:39.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:39 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4166731171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:39.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:56:39.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:39 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4166731171' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:39.982 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:40 smithi139 ceph-mon[33027]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:40 smithi096 ceph-mon[29288]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:40.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:40 smithi097 ceph-mon[32667]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 -- 172.21.15.96:0/3035503486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c1011f0 msgr2=0x7f760c1015d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 --2- 172.21.15.96:0/3035503486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c1011f0 0x7f760c1015d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f76080099c0 tx=0x7f7608009cd0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 -- 172.21.15.96:0/3035503486 shutdown_connections 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 --2- 172.21.15.96:0/3035503486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c107e10 0x7f760c10f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 --2- 172.21.15.96:0/3035503486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c1011f0 0x7f760c1015d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 --2- 172.21.15.96:0/3035503486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f760c101ba0 0x7f760c1076e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 -- 172.21.15.96:0/3035503486 >> 172.21.15.96:0/3035503486 conn(0x7f760c077f40 msgr2=0x7f760c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:41.604 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 -- 172.21.15.96:0/3035503486 shutdown_connections 2024-04-25T20:56:41.605 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 -- 172.21.15.96:0/3035503486 wait complete. 2024-04-25T20:56:41.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 Processor -- start 2024-04-25T20:56:41.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.603+0000 7f7613b12700 1 -- start start 2024-04-25T20:56:41.607 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f760c1011f0 0x7f760c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 0x7f760c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c107e10 0x7f760c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f760c112d30 con 0x7f760c101ba0 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f760c112bb0 con 0x7f760c1011f0 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f760c112eb0 con 0x7f760c107e10 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 0x7f760c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 0x7f760c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55710/0 (socket says 172.21.15.96:55710) 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 -- 172.21.15.96:0/3164298649 learned_addr learned my addr 172.21.15.96:0/3164298649 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76118ae700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f760c1011f0 0x7f760c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76120af700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c107e10 0x7f760c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 -- 172.21.15.96:0/3164298649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c107e10 msgr2=0x7f760c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c107e10 0x7f760c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 -- 172.21.15.96:0/3164298649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f760c1011f0 msgr2=0x7f760c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f760c1011f0 0x7f760c199200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.608 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 -- 172.21.15.96:0/3164298649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c19e210 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f76110ad700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 0x7f760c199740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00b4d0 tx=0x7f75fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc015020 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75fc011ce0 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c19e500 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.604+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75fc010cd0 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.605+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c1a5d80 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.605+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c069730 con 0x7f760c101ba0 2024-04-25T20:56:41.609 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.606+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f75fc018020 con 0x7f760c101ba0 2024-04-25T20:56:41.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.607+0000 7f7602ffd700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f75f80a3cc0 0x7f75f80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:41.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.607+0000 7f76118ae700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f75f80a3cc0 0x7f75f80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:41.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.607+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f75fc0d00c0 con 0x7f760c101ba0 2024-04-25T20:56:41.610 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.607+0000 7f76118ae700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f75f80a3cc0 0x7f75f80a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7608009750 tx=0x7f760801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:41.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.611+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f75fc01f020 con 0x7f760c101ba0 2024-04-25T20:56:41.811 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.810+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f760c02df80 con 0x7f760c101ba0 2024-04-25T20:56:41.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.812+0000 7f7602ffd700 1 -- 172.21.15.96:0/3164298649 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22523 (secure 0 0 0) 0x7f75fc09cae0 con 0x7f760c101ba0 2024-04-25T20:56:41.813 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:41.813 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":93,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:55:10.320396+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f75f80a3cc0 msgr2=0x7f75f80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f75f80a3cc0 0x7f75f80a6180 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7608009750 tx=0x7f760801a040 comp rx=0 tx=0).stop 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 msgr2=0x7f760c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 0x7f760c199740 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f75fc00b4d0 tx=0x7f75fc00b890 comp rx=0 tx=0).stop 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 shutdown_connections 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f75f80a3cc0 0x7f75f80a6180 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f760c107e10 0x7f760c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f760c1011f0 0x7f760c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 --2- 172.21.15.96:0/3164298649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f760c101ba0 0x7f760c199740 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 >> 172.21.15.96:0/3164298649 conn(0x7f760c077f40 msgr2=0x7f760c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 shutdown_connections 2024-04-25T20:56:41.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:41.814+0000 7f7613b12700 1 -- 172.21.15.96:0/3164298649 wait complete. 2024-04-25T20:56:42.432 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:42 smithi096 ceph-mon[29288]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:42.433 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:42 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3164298649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:56:42.460 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:56:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:42 smithi139 ceph-mon[33027]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:42.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:42 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3164298649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:56:42.700 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:42 smithi097 ceph-mon[32667]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:42 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3164298649' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:56:44.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 -- 172.21.15.96:0/711979540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480108a70 msgr2=0x7f2480108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 --2- 172.21.15.96:0/711979540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480108a70 0x7f2480108e50 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2470009990 tx=0x7f2470009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 -- 172.21.15.96:0/711979540 shutdown_connections 2024-04-25T20:56:44.285 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 --2- 172.21.15.96:0/711979540 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 0x7f248010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 --2- 172.21.15.96:0/711979540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480108a70 0x7f2480108e50 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 --2- 172.21.15.96:0/711979540 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480102ac0 0x7f2480102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 -- 172.21.15.96:0/711979540 >> 172.21.15.96:0/711979540 conn(0x7f2480079bf0 msgr2=0x7f248007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 -- 172.21.15.96:0/711979540 shutdown_connections 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.282+0000 7f2484a10700 1 -- 172.21.15.96:0/711979540 wait complete. 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 Processor -- start 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 -- start start 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 0x7f2480199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 0x7f24801996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480108a70 0x7f248019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112c70 con 0x7f2480102ac0 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112af0 con 0x7f2480103460 2024-04-25T20:56:44.286 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f2484a10700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2480112df0 con 0x7f2480108a70 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f247e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 0x7f2480199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f247e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 0x7f2480199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55716/0 (socket says 172.21.15.96:55716) 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.283+0000 7f247e59c700 1 -- 172.21.15.96:0/2334357239 learned_addr learned my addr 172.21.15.96:0/2334357239 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2475d9b700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 0x7f24801996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247ed9d700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480108a70 0x7f248019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247e59c700 1 -- 172.21.15.96:0/2334357239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480108a70 msgr2=0x7f248019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247e59c700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480108a70 0x7f248019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247e59c700 1 -- 172.21.15.96:0/2334357239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 msgr2=0x7f24801996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247e59c700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 0x7f24801996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247e59c700 1 -- 172.21.15.96:0/2334357239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f248019e120 con 0x7f2480102ac0 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2475d9b700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 0x7f24801996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247ed9d700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480108a70 0x7f248019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f247e59c700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 0x7f2480199160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f24700057f0 tx=0x7f2470003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:44.287 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470027070 con 0x7f2480102ac0 2024-04-25T20:56:44.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24700041c0 con 0x7f2480102ac0 2024-04-25T20:56:44.288 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f248019e3b0 con 0x7f2480102ac0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470022e60 con 0x7f2480102ac0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.284+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2480074c70 con 0x7f2480102ac0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.285+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24801041b0 con 0x7f2480102ac0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.286+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f247002a070 con 0x7f2480102ac0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.287+0000 7f2477fff700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f246c0a8050 0x7f246c0aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.287+0000 7f2475d9b700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f246c0a8050 0x7f246c0aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.287+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f24700d1640 con 0x7f2480102ac0 2024-04-25T20:56:44.289 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.287+0000 7f2475d9b700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f246c0a8050 0x7f246c0aa510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2468005fd0 tx=0x7f2468005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:44.291 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.290+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2470013070 con 0x7f2480102ac0 2024-04-25T20:56:44.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.534+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f248002df80 con 0x7f2480102ac0 2024-04-25T20:56:44.536 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.535+0000 7f2477fff700 1 -- 172.21.15.96:0/2334357239 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f247009e150 con 0x7f2480102ac0 2024-04-25T20:56:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:56:44.537 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:54:49.357397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:56:44.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f246c0a8050 msgr2=0x7f246c0aa510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f246c0a8050 0x7f246c0aa510 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2468005fd0 tx=0x7f2468005e30 comp rx=0 tx=0).stop 2024-04-25T20:56:44.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 msgr2=0x7f2480199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:44.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 0x7f2480199160 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f24700057f0 tx=0x7f2470003730 comp rx=0 tx=0).stop 2024-04-25T20:56:44.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 shutdown_connections 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f246c0a8050 0x7f246c0aa510 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2480108a70 0x7f248019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2480102ac0 0x7f2480199160 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 --2- 172.21.15.96:0/2334357239 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2480103460 0x7f24801996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 >> 172.21.15.96:0/2334357239 conn(0x7f2480079bf0 msgr2=0x7f2480100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 shutdown_connections 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:44.537+0000 7f2484a10700 1 -- 172.21.15.96:0/2334357239 wait complete. 2024-04-25T20:56:44.539 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:56:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:44 smithi139 ceph-mon[33027]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:44 smithi096 ceph-mon[29288]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:44.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:44 smithi097 ceph-mon[32667]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:45.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T20:54:31.232491+0000', 'modified': '2024-04-25T20:54:49.357397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24443, 'mds_1': 24442, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24398': {'gid': 24398, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/122204709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 122204709}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 122204709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6835/1172572610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1172572610}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 1172572610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/57160731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 57160731}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 57160731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.97:6837/16128615', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 16128615}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 16128615}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.139:6839/2573809872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 2573809872}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 2573809872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.97:6839/3461334961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3461334961}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3461334961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24443, 'qdb_cluster': [24442, 24422, 24443]} 2024-04-25T20:56:45.223 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T20:56:45.224 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T20:56:45.466 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:45.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:45 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2334357239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:45.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:45 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2334357239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:45.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:45 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2334357239' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:56:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:46 smithi096 ceph-mon[29288]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:46.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:46 smithi097 ceph-mon[32667]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:46 smithi139 ceph-mon[33027]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:47.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 -- 172.21.15.96:0/2186945512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 msgr2=0x7f73d0102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/2186945512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0102f80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f73bc009990 tx=0x7f73bc009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 -- 172.21.15.96:0/2186945512 shutdown_connections 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/2186945512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 0x7f73d010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/2186945512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 0x7f73d0108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/2186945512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0102f80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 -- 172.21.15.96:0/2186945512 >> 172.21.15.96:0/2186945512 conn(0x7f73d0079c00 msgr2=0x7f73d007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 -- 172.21.15.96:0/2186945512 shutdown_connections 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.148+0000 7f73d6a9c700 1 -- 172.21.15.96:0/2186945512 wait complete. 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 Processor -- start 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 -- start start 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 0x7f73d0075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 0x7f73d00757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0112c50 con 0x7f73d0102b00 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0112ad0 con 0x7f73d01034c0 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d6a9c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73d0112dd0 con 0x7f73d0108a10 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d4838700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d4838700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50302/0 (socket says 172.21.15.96:50302) 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73d4838700 1 -- 172.21.15.96:0/3664747991 learned_addr learned my addr 172.21.15.96:0/3664747991 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.149+0000 7f73cffff700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 0x7f73d0075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d5039700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 0x7f73d00757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d4838700 1 -- 172.21.15.96:0/3664747991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 msgr2=0x7f73d00757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d4838700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 0x7f73d00757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d4838700 1 -- 172.21.15.96:0/3664747991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 msgr2=0x7f73d0075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d4838700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 0x7f73d0075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d4838700 1 -- 172.21.15.96:0/3664747991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d01a76f0 con 0x7f73d0102b00 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d5039700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 0x7f73d00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:47.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73cffff700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 0x7f73d0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:47.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d4838700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0074d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f73c400b530 tx=0x7f73c400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:47.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73cdffb700 1 -- 172.21.15.96:0/3664747991 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4015020 con 0x7f73d0102b00 2024-04-25T20:56:47.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d01a79e0 con 0x7f73d0102b00 2024-04-25T20:56:47.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73cdffb700 1 -- 172.21.15.96:0/3664747991 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c4011ce0 con 0x7f73d0102b00 2024-04-25T20:56:47.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73cdffb700 1 -- 172.21.15.96:0/3664747991 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c4010d20 con 0x7f73d0102b00 2024-04-25T20:56:47.154 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.150+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d01a7f20 con 0x7f73d0102b00 2024-04-25T20:56:47.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.153+0000 7f73cdffb700 1 -- 172.21.15.96:0/3664747991 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73c4018020 con 0x7f73d0102b00 2024-04-25T20:56:47.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.153+0000 7f73cdffb700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.153+0000 7f73cffff700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.153+0000 7f73cdffb700 1 -- 172.21.15.96:0/3664747991 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f73c40cfb90 con 0x7f73d0102b00 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73cffff700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f73bc006010 tx=0x7f73bc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 Processor -- start 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 -- start start 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7077d0 0x55d88f707bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f70c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f70c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d88f659ce0 con 0x55d88f70c580 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d88f600910 con 0x55d88f7080f0 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73e1325b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d88f655ca0 con 0x55d88f7077d0 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.154+0000 7f73d5039700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f70c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f70c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50314/0 (socket says 172.21.15.96:50314) 2024-04-25T20:56:47.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 -- 172.21.15.96:0/3216058134 learned_addr learned my addr 172.21.15.96:0/3216058134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d4838700 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7077d0 0x55d88f707bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73cffff700 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f70c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 -- 172.21.15.96:0/3216058134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7077d0 msgr2=0x55d88f707bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7077d0 0x55d88f707bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 -- 172.21.15.96:0/3216058134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 msgr2=0x55d88f70c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f70c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 -- 172.21.15.96:0/3216058134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88f5a7880 con 0x55d88f70c580 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73d5039700 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f70c9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f73c80084f0 tx=0x7f73c8008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3af5bbb4ca48e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73baffd700 1 -- 172.21.15.96:0/3216058134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8018070 con 0x55d88f70c580 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73baffd700 1 -- 172.21.15.96:0/3216058134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c8011bc0 con 0x55d88f70c580 2024-04-25T20:56:47.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.155+0000 7f73baffd700 1 -- 172.21.15.96:0/3216058134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8013b50 con 0x55d88f70c580 2024-04-25T20:56:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 -- 172.21.15.96:0/3216058134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 msgr2=0x55d88f70c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f70c9e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f73c80084f0 tx=0x7f73c8008800 comp rx=0 tx=0).stop 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 -- 172.21.15.96:0/3216058134 shutdown_connections 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f70c9e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7077d0 0x55d88f707bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3216058134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f70c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 -- 172.21.15.96:0/3216058134 >> 172.21.15.96:0/3216058134 conn(0x55d88f634640 msgr2=0x55d88f702820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 -- 172.21.15.96:0/3216058134 shutdown_connections 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 -- 172.21.15.96:0/3216058134 wait complete. 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 -- start start 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.156+0000 7f73e1325b80 1 Processor -- start 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 0x55d88f713a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f710d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f7112b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d88f714610 con 0x55d88f7077d0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d88f5f8b80 con 0x55d88f7080f0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d88f714790 con 0x55d88f70c580 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73e1325b80 10 client.0 _ll_register_callbacks cb 0x55d88f5f2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 0x55d88f713a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 0x55d88f713a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50330/0 (socket says 172.21.15.96:50330) 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 -- 172.21.15.96:0/2683903785 learned_addr learned my addr 172.21.15.96:0/2683903785 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73cffff700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f710d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d5039700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f7112b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 -- 172.21.15.96:0/2683903785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 msgr2=0x55d88f7112b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f7112b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 -- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 msgr2=0x55d88f710d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f710d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 -- 172.21.15.96:0/2683903785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88f611220 con 0x55d88f7077d0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73d4838700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 0x55d88f713a70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f73c400bcf0 tx=0x7f73c409c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ed92b16abd3ed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:47.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.157+0000 7f73affff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73affff700 1 -- 172.21.15.96:0/2683903785 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c40b5420 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73affff700 1 -- 172.21.15.96:0/2683903785 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f73c409c170 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c8011bc0 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73affff700 1 -- 172.21.15.96:0/2683903785 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73c40a02c0 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 10 client.14970 fetch_fsmap learned FSMap version 25 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d88f71ef20 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73affff700 1 -- 172.21.15.96:0/2683903785 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c8011bc0 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73affff700 1 -- 172.21.15.96:0/2683903785 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f73c4011830 con 0x55d88f7077d0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 10 client.14970 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 20 client.14970 populate_metadata read hostname 'smithi096' 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f7214c0 0x55d88f722930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 4 client.14970 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d88f6674d0 con 0x55d88f7214c0 2024-04-25T20:56:47.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.158+0000 7f73cffff700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f7214c0 0x55d88f722930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.159+0000 7f73cffff700 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f7214c0 0x55d88f722930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=92608accb6f167f3 server_cookie=98c6953b47c6fdf4 in_seq=0 out_seq=0 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.159+0000 7f73affff700 10 client.14970 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73affff700 1 -- 172.21.15.96:0/2683903785 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d88f6674d0 con 0x55d88f7214c0 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73affff700 10 client.14970 handle_command_reply: tid=0 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 client.14970 shutdown 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 msgr2=0x55d88f713a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 0x55d88f713a70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f73c400bcf0 tx=0x7f73c409c6c0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 shutdown_connections 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f7214c0 0x55d88f722930 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f7112b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7077d0 0x55d88f713a70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 --2- 172.21.15.96:0/2683903785 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7080f0 0x55d88f710d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 >> 172.21.15.96:0/2683903785 conn(0x55d88f634640 msgr2=0x55d88f701a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73affff700 0 client.14970 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:56:47.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 shutdown_connections 2024-04-25T20:56:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 1 -- 172.21.15.96:0/2683903785 wait complete. 2024-04-25T20:56:47.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.160+0000 7f73e1325b80 20 client.14970 trim_cache size 0 max 16384 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 Processor -- start 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 -- start start 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f7135b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7080f0 0x55d88f702210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f702750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d88f720c70 con 0x55d88f70c580 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d88f721340 con 0x55d88f7077d0 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73e1325b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d88f721a10 con 0x55d88f7080f0 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f702750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f702750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50338/0 (socket says 172.21.15.96:50338) 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 -- 172.21.15.96:0/3903934481 learned_addr learned my addr 172.21.15.96:0/3903934481 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 -- 172.21.15.96:0/3903934481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7080f0 msgr2=0x55d88f702210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7080f0 0x55d88f702210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 -- 172.21.15.96:0/3903934481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 msgr2=0x55d88f7135b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f7135b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.180+0000 7f73d5039700 1 -- 172.21.15.96:0/3903934481 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d88f714740 con 0x55d88f70c580 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73d5039700 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f702750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f73c8000f80 tx=0x7f73c80076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17155f4e1bf2cfe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73ba7fc700 1 -- 172.21.15.96:0/3903934481 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8018040 con 0x55d88f70c580 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73ba7fc700 1 -- 172.21.15.96:0/3903934481 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c8013c50 con 0x55d88f70c580 2024-04-25T20:56:47.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73ba7fc700 1 -- 172.21.15.96:0/3903934481 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73c8011ed0 con 0x55d88f70c580 2024-04-25T20:56:47.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 -- 172.21.15.96:0/3903934481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 msgr2=0x55d88f702750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f702750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f73c8000f80 tx=0x7f73c80076b0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 -- 172.21.15.96:0/3903934481 shutdown_connections 2024-04-25T20:56:47.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f70c580 0x55d88f702750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f7135b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 --2- 172.21.15.96:0/3903934481 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f7080f0 0x55d88f702210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 -- 172.21.15.96:0/3903934481 >> 172.21.15.96:0/3903934481 conn(0x55d88f634640 msgr2=0x55d88f706e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 -- 172.21.15.96:0/3903934481 shutdown_connections 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.181+0000 7f73e1325b80 1 -- 172.21.15.96:0/3903934481 wait complete. 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 -- start start 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 Processor -- start 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f70e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 0x55d88f70e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f72a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d88f7227b0 con 0x55d88f7080f0 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d88f7220e0 con 0x55d88f7077d0 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d88f722e80 con 0x55d88f70c580 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73e1325b80 10 client.0 _ll_register_callbacks cb 0x55d88f5f2900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73cffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 0x55d88f70e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73cffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 0x55d88f70e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50350/0 (socket says 172.21.15.96:50350) 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73cffff700 1 -- 172.21.15.96:0/1331527959 learned_addr learned my addr 172.21.15.96:0/1331527959 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:47.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73d4838700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f70e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.182+0000 7f73d5039700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f72a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73cffff700 1 -- 172.21.15.96:0/1331527959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 msgr2=0x55d88f72a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73cffff700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f72a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73cffff700 1 -- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 msgr2=0x55d88f70e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73cffff700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f70e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73cffff700 1 -- 172.21.15.96:0/1331527959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d88f72a920 con 0x55d88f7080f0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73d4838700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f70e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73d5039700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f72a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73cffff700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 0x55d88f70e5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73bc000c00 tx=0x7f73bc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8091f4f55a8c7d3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73affff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73affff700 1 -- 172.21.15.96:0/1331527959 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc03b060 con 0x55d88f7080f0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73affff700 1 -- 172.21.15.96:0/1331527959 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f73bc003c70 con 0x55d88f7080f0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73c8013c50 con 0x55d88f7080f0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73affff700 1 -- 172.21.15.96:0/1331527959 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f73bc00be60 con 0x55d88f7080f0 2024-04-25T20:56:47.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 10 client.14976 fetch_fsmap learned FSMap version 25 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d88f788e60 con 0x55d88f7080f0 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73affff700 1 -- 172.21.15.96:0/1331527959 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73c8013c50 con 0x55d88f7080f0 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73affff700 1 -- 172.21.15.96:0/1331527959 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f73bc040070 con 0x55d88f7080f0 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 10 client.14976 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 10 client.14976 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 20 client.14976 populate_metadata read hostname 'smithi096' 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f78da20 0x55d88f78c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 4 client.14976 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.183+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "status"}) -- 0x55d88f661570 con 0x55d88f78da20 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73d4838700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f78da20 0x55d88f78c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73affff700 10 client.14976 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73d4838700 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f78da20 0x55d88f78c730 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49097c13f73d6b4c server_cookie=7014008b22110463 in_seq=0 out_seq=0 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73affff700 1 -- 172.21.15.96:0/1331527959 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d88f661570 con 0x55d88f78da20 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73affff700 10 client.14976 handle_command_reply: tid=0 2024-04-25T20:56:47.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 client.14976 shutdown 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 msgr2=0x55d88f70e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 0x55d88f70e5e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f73bc000c00 tx=0x7f73bc009310 comp rx=0 tx=0).stop 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 shutdown_connections 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55d88f78da20 0x55d88f78c730 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d88f70c580 0x55d88f72a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d88f7077d0 0x55d88f70e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73affff700 0 client.14976 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 --2- 172.21.15.96:0/1331527959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d88f7080f0 0x55d88f70e5e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.184+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 >> 172.21.15.96:0/1331527959 conn(0x55d88f634640 msgr2=0x55d88f70af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 shutdown_connections 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73e1325b80 1 -- 172.21.15.96:0/1331527959 wait complete. 2024-04-25T20:56:47.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73e1325b80 20 client.14976 trim_cache size 0 max 16384 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24422, 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 122.859617075, 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 93, 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T20:56:47.192 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 166.52166750200001 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 msgr2=0x7f73c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f73bc006010 tx=0x7f73bc00b540 comp rx=0 tx=0).stop 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 msgr2=0x7f73d0074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0074d40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f73c400b530 tx=0x7f73c400b8f0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 shutdown_connections 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d0108a10 0x7f73d00757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d0102b00 0x7f73d0074d40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 --2- 172.21.15.96:0/3664747991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d01034c0 0x7f73d0075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 >> 172.21.15.96:0/3664747991 conn(0x7f73d0079c00 msgr2=0x7f73d0100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 shutdown_connections 2024-04-25T20:56:47.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:47.185+0000 7f73d6a9c700 1 -- 172.21.15.96:0/3664747991 wait complete. 2024-04-25T20:56:47.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T20:56:48.190 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:48.648 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:48 smithi096 ceph-mon[29288]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:48 smithi097 ceph-mon[32667]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:48 smithi139 ceph-mon[33027]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 -- 172.21.15.96:0/1171499311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368102300 msgr2=0x7fc36810e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 --2- 172.21.15.96:0/1171499311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368102300 0x7fc36810e7f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc36000b0a0 tx=0x7fc36000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 -- 172.21.15.96:0/1171499311 shutdown_connections 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 --2- 172.21.15.96:0/1171499311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368102300 0x7fc36810e7f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 --2- 172.21.15.96:0/1171499311 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368107990 0x7fc368107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 --2- 172.21.15.96:0/1171499311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368101940 0x7fc368101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 -- 172.21.15.96:0/1171499311 >> 172.21.15.96:0/1171499311 conn(0x7fc3680788f0 msgr2=0x7fc368078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 -- 172.21.15.96:0/1171499311 shutdown_connections 2024-04-25T20:56:49.840 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.837+0000 7fc36e10c700 1 -- 172.21.15.96:0/1171499311 wait complete. 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 Processor -- start 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 -- start start 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368101940 0x7fc36819a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368102300 0x7fc36819a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 0x7fc3681a5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc368111bd0 con 0x7fc368107990 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc368111a50 con 0x7fc368102300 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc36e10c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc368111d50 con 0x7fc368101940 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 0x7fc3681a5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 0x7fc3681a5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50366/0 (socket says 172.21.15.96:50366) 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 -- 172.21.15.96:0/3829739053 learned_addr learned my addr 172.21.15.96:0/3829739053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc35effd700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368102300 0x7fc36819a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc3677fe700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368101940 0x7fc36819a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 -- 172.21.15.96:0/3829739053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368101940 msgr2=0x7fc36819a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368101940 0x7fc36819a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 -- 172.21.15.96:0/3829739053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368102300 msgr2=0x7fc36819a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368102300 0x7fc36819a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc367fff700 1 -- 172.21.15.96:0/3829739053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3681a5650 con 0x7fc368107990 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc35effd700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368102300 0x7fc36819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.838+0000 7fc3677fe700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368101940 0x7fc36819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.839+0000 7fc367fff700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 0x7fc3681a5110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3600062a0 tx=0x7fc360004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.839+0000 7fc3657fa700 1 -- 172.21.15.96:0/3829739053 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc360021070 con 0x7fc368107990 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.839+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3681a5820 con 0x7fc368107990 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.839+0000 7fc3657fa700 1 -- 172.21.15.96:0/3829739053 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc360007d00 con 0x7fc368107990 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.839+0000 7fc3657fa700 1 -- 172.21.15.96:0/3829739053 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36001d450 con 0x7fc368107990 2024-04-25T20:56:49.842 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.839+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3681a5cd0 con 0x7fc368107990 2024-04-25T20:56:49.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.841+0000 7fc3657fa700 1 -- 172.21.15.96:0/3829739053 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc360004020 con 0x7fc368107990 2024-04-25T20:56:49.844 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.842+0000 7fc3657fa700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc3540a3c40 0x7fc3540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.842+0000 7fc3677fe700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc3540a3c40 0x7fc3540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.842+0000 7fc3657fa700 1 -- 172.21.15.96:0/3829739053 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fc3600d1570 con 0x7fc368107990 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.842+0000 7fc3677fe700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc3540a3c40 0x7fc3540a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc358007900 tx=0x7fc358008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.842+0000 7fc378995b80 1 Processor -- start 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.842+0000 7fc378995b80 1 -- start start 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc378995b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc378995b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc378995b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc378995b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610e772fc40 con 0x5610e77dc0b0 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc378995b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5610e76d68e0 con 0x5610e77e7880 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc378995b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5610e772bc00 con 0x5610e77dc9d0 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50382/0 (socket says 172.21.15.96:50382) 2024-04-25T20:56:49.845 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 -- 172.21.15.96:0/1864103625 learned_addr learned my addr 172.21.15.96:0/1864103625 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc35effd700 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 -- 172.21.15.96:0/1864103625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 msgr2=0x5610e77e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 -- 172.21.15.96:0/1864103625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 msgr2=0x5610e77e9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc367fff700 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 -- 172.21.15.96:0/1864103625 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e769a6f0 con 0x5610e77dc0b0 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77dc490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c910 tx=0x7fc35800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0afbae4ff71f79f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc35dffb700 1 -- 172.21.15.96:0/1864103625 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358023070 con 0x5610e77dc0b0 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc35dffb700 1 -- 172.21.15.96:0/1864103625 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc358007e40 con 0x5610e77dc0b0 2024-04-25T20:56:49.846 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.843+0000 7fc35dffb700 1 -- 172.21.15.96:0/1864103625 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc35801ee10 con 0x5610e77dc0b0 2024-04-25T20:56:49.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 -- 172.21.15.96:0/1864103625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 msgr2=0x5610e77dc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77dc490 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fc35800c910 tx=0x7fc35800f000 comp rx=0 tx=0).stop 2024-04-25T20:56:49.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 -- 172.21.15.96:0/1864103625 shutdown_connections 2024-04-25T20:56:49.850 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77dc490 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 --2- 172.21.15.96:0/1864103625 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 -- 172.21.15.96:0/1864103625 >> 172.21.15.96:0/1864103625 conn(0x5610e7709840 msgr2=0x5610e77d8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 -- 172.21.15.96:0/1864103625 shutdown_connections 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.844+0000 7fc378995b80 1 -- 172.21.15.96:0/1864103625 wait complete. 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 -- start start 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 Processor -- start 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77e6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610e77ea370 con 0x5610e77dc0b0 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5610e76e9560 con 0x5610e77e7880 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5610e77ea4f0 con 0x5610e77dc9d0 2024-04-25T20:56:49.851 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc378995b80 10 client.0 _ll_register_callbacks cb 0x5610e76c88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77e6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77e6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50398/0 (socket says 172.21.15.96:50398) 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 -- 172.21.15.96:0/1743519644 learned_addr learned my addr 172.21.15.96:0/1743519644 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc35effd700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc367fff700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 -- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 msgr2=0x5610e77e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 -- 172.21.15.96:0/1743519644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 msgr2=0x5610e77e4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc3677fe700 1 -- 172.21.15.96:0/1743519644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e767d880 con 0x5610e77dc0b0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.845+0000 7fc367fff700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77e6f50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3580048a0 tx=0x7fc35801e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29a57221b365a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc346ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:56:49.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc346ffd700 1 -- 172.21.15.96:0/1743519644 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358023040 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc346ffd700 1 -- 172.21.15.96:0/1743519644 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fc3580137b0 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5610e773d430 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc346ffd700 1 -- 172.21.15.96:0/1743519644 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc35801ee40 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 10 client.14988 fetch_fsmap learned FSMap version 25 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5610e77f4bf0 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc346ffd700 1 -- 172.21.15.96:0/1743519644 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5610e773d430 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc346ffd700 1 -- 172.21.15.96:0/1743519644 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc358026030 con 0x5610e77dc0b0 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 10 client.14988 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 20 client.14988 populate_metadata read hostname 'smithi096' 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e77f6e50 0x5610e77f82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 4 client.14988 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.846+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610e77374d0 con 0x5610e77f6e50 2024-04-25T20:56:49.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.847+0000 7fc35effd700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e77f6e50 0x5610e77f82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.847+0000 7fc346ffd700 10 client.14988 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.847+0000 7fc35effd700 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e77f6e50 0x5610e77f82c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a92959790ec10ca server_cookie=9015b41e41740ba8 in_seq=0 out_seq=0 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc346ffd700 1 -- 172.21.15.96:0/1743519644 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610e77374d0 con 0x5610e77f6e50 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc346ffd700 10 client.14988 handle_command_reply: tid=0 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 client.14988 shutdown 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 msgr2=0x5610e77e6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77e6f50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3580048a0 tx=0x7fc35801e890 comp rx=0 tx=0).stop 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 shutdown_connections 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e77f6e50 0x5610e77f82c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77e7880 0x5610e77e4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e77e6f50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 --2- 172.21.15.96:0/1743519644 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc9d0 0x5610e77e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 >> 172.21.15.96:0/1743519644 conn(0x5610e7709840 msgr2=0x5610e77d81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc346ffd700 0 client.14988 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T20:56:49.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 shutdown_connections 2024-04-25T20:56:49.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.848+0000 7fc378995b80 1 -- 172.21.15.96:0/1743519644 wait complete. 2024-04-25T20:56:49.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.849+0000 7fc378995b80 20 client.14988 trim_cache size 0 max 16384 2024-04-25T20:56:49.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 Processor -- start 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 -- start start 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc0b0 0x5610e7801b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e77e15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 0x5610e77e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610e77f6600 con 0x5610e77e7880 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5610e77f6cd0 con 0x5610e77dc9d0 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc378995b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5610e77f73a0 con 0x5610e77dc0b0 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 0x5610e77e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 0x5610e77e1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50404/0 (socket says 172.21.15.96:50404) 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 -- 172.21.15.96:0/1818444036 learned_addr learned my addr 172.21.15.96:0/1818444036 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc0b0 0x5610e7801b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc35effd700 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e77e15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 -- 172.21.15.96:0/1818444036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc0b0 msgr2=0x5610e7801b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc0b0 0x5610e7801b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 -- 172.21.15.96:0/1818444036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 msgr2=0x5610e77e15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e77e15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 -- 172.21.15.96:0/1818444036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610e77e28b0 con 0x5610e77e7880 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc3677fe700 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc0b0 0x5610e7801b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.869+0000 7fc367fff700 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 0x5610e77e1b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3600cf110 tx=0x7fc360009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0aae165e2ca55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc35d7fa700 1 -- 172.21.15.96:0/1818444036 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36002a030 con 0x5610e77e7880 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc35d7fa700 1 -- 172.21.15.96:0/1818444036 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3600a1d30 con 0x5610e77e7880 2024-04-25T20:56:49.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc35d7fa700 1 -- 172.21.15.96:0/1818444036 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc36009dbe0 con 0x5610e77e7880 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 -- 172.21.15.96:0/1818444036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 msgr2=0x5610e77e1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 0x5610e77e1b20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3600cf110 tx=0x7fc360009040 comp rx=0 tx=0).stop 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 -- 172.21.15.96:0/1818444036 shutdown_connections 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77e7880 0x5610e77e1b20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77dc0b0 0x5610e7801b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 --2- 172.21.15.96:0/1818444036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e77e15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.870+0000 7fc378995b80 1 -- 172.21.15.96:0/1818444036 >> 172.21.15.96:0/1818444036 conn(0x5610e7709840 msgr2=0x5610e77d7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 -- 172.21.15.96:0/1818444036 shutdown_connections 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 -- 172.21.15.96:0/1818444036 wait complete. 2024-04-25T20:56:49.876 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 -- start start 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 Processor -- start 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e7856120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e7856660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77e7880 0x5610e7860fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610e77f8140 con 0x5610e77dc0b0 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5610e77f7a70 con 0x5610e77dc9d0 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5610e77f8810 con 0x5610e77e7880 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc378995b80 10 client.0 _ll_register_callbacks cb 0x5610e76c88d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.871+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e7856120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e7856120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50406/0 (socket says 172.21.15.96:50406) 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 -- 172.21.15.96:0/2489713904 learned_addr learned my addr 172.21.15.96:0/2489713904 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc367fff700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77e7880 0x5610e7860fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc35effd700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e7856660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 -- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77e7880 msgr2=0x5610e7860fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77e7880 0x5610e7860fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 -- 172.21.15.96:0/2489713904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 msgr2=0x5610e7856660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e7856660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 -- 172.21.15.96:0/2489713904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610e7861630 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc3677fe700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e7856120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc358007540 tx=0x7fc3580296f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c9c53c1a441dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc346ffd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc346ffd700 1 -- 172.21.15.96:0/2489713904 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc358023040 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc346ffd700 1 -- 172.21.15.96:0/2489713904 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fc3580139b0 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3600a1d30 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc346ffd700 1 -- 172.21.15.96:0/2489713904 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc358013bb0 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc378995b80 10 client.14994 fetch_fsmap learned FSMap version 25 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.872+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5610e7862930 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc346ffd700 1 -- 172.21.15.96:0/2489713904 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3600a1d30 con 0x5610e77dc0b0 2024-04-25T20:56:49.878 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc346ffd700 1 -- 172.21.15.96:0/2489713904 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc358026070 con 0x5610e77dc0b0 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc378995b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc378995b80 10 client.14994 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc378995b80 20 client.14994 populate_metadata read hostname 'smithi096' 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc378995b80 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e7867450 0x5610e7866160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc378995b80 4 client.14994 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "status"}) -- 0x5610e75b0200 con 0x5610e7867450 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc35effd700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e7867450 0x5610e7866160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc346ffd700 10 client.14994 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.873+0000 7fc35effd700 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e7867450 0x5610e7866160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f1999eb372b7922 server_cookie=6899918251db491d in_seq=0 out_seq=0 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc346ffd700 1 -- 172.21.15.96:0/2489713904 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5610e75b0200 con 0x5610e7867450 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc346ffd700 10 client.14994 handle_command_reply: tid=0 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 client.14994 shutdown 2024-04-25T20:56:49.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 msgr2=0x5610e7856120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e7856120 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc358007540 tx=0x7fc3580296f0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 shutdown_connections 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5610e77e7880 0x5610e7860fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610e77dc0b0 0x5610e7856120 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x5610e7867450 0x5610e7866160 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 --2- 172.21.15.96:0/2489713904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5610e77dc9d0 0x5610e7856660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 >> 172.21.15.96:0/2489713904 conn(0x5610e7709840 msgr2=0x5610e77e5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc346ffd700 0 client.14994 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 shutdown_connections 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 1 -- 172.21.15.96:0/2489713904 wait complete. 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.874+0000 7fc378995b80 20 client.14994 trim_cache size 0 max 16384 2024-04-25T20:56:49.880 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24442, 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 126.558598625, 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 93, 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 166.450457486 2024-04-25T20:56:49.881 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:56:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc3540a3c40 msgr2=0x7fc3540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc3540a3c40 0x7fc3540a6100 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc358007900 tx=0x7fc358008040 comp rx=0 tx=0).stop 2024-04-25T20:56:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 msgr2=0x7fc3681a5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 0x7fc3681a5110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fc3600062a0 tx=0x7fc360004ed0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 shutdown_connections 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fc3540a3c40 0x7fc3540a6100 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc368107990 0x7fc3681a5110 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc368101940 0x7fc36819a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 --2- 172.21.15.96:0/3829739053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc368102300 0x7fc36819a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 >> 172.21.15.96:0/3829739053 conn(0x7fc3680788f0 msgr2=0x7fc3680ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 shutdown_connections 2024-04-25T20:56:49.883 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:49.875+0000 7fc36e10c700 1 -- 172.21.15.96:0/3829739053 wait complete. 2024-04-25T20:56:50.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:50 smithi096 ceph-mon[29288]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:50.566 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T20:56:50.806 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:50.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:50 smithi097 ceph-mon[32667]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:50.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:50 smithi139 ceph-mon[33027]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:56:52.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 -- 172.21.15.96:0/3730040887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4101940 msgr2=0x7f28c4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 --2- 172.21.15.96:0/3730040887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4101940 0x7f28c4101dc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009990 tx=0x7f28b0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 -- 172.21.15.96:0/3730040887 shutdown_connections 2024-04-25T20:56:52.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 --2- 172.21.15.96:0/3730040887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4102300 0x7f28c410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 --2- 172.21.15.96:0/3730040887 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4107990 0x7f28c4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 --2- 172.21.15.96:0/3730040887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4101940 0x7f28c4101dc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 -- 172.21.15.96:0/3730040887 >> 172.21.15.96:0/3730040887 conn(0x7f28c40788f0 msgr2=0x7f28c4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 -- 172.21.15.96:0/3730040887 shutdown_connections 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.469+0000 7f28caba5700 1 -- 172.21.15.96:0/3730040887 wait complete. 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 Processor -- start 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 -- start start 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4101940 0x7f28c4199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 0x7f28c4199760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4107990 0x7f28c419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4111b60 con 0x7f28c4102300 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f28c41119e0 con 0x7f28c4107990 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28caba5700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f28c4111ce0 con 0x7f28c4101940 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 0x7f28c4199760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.479 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 0x7f28c4199760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50432/0 (socket says 172.21.15.96:50432) 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.470+0000 7f28c3fff700 1 -- 172.21.15.96:0/1955550114 learned_addr learned my addr 172.21.15.96:0/1955550114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c9142700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4107990 0x7f28c419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c8941700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4101940 0x7f28c4199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c3fff700 1 -- 172.21.15.96:0/1955550114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4101940 msgr2=0x7f28c4199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c3fff700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4101940 0x7f28c4199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c3fff700 1 -- 172.21.15.96:0/1955550114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4107990 msgr2=0x7f28c419daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c3fff700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4107990 0x7f28c419daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c3fff700 1 -- 172.21.15.96:0/1955550114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c419e1e0 con 0x7f28c4102300 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c9142700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4107990 0x7f28c419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c8941700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4101940 0x7f28c4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c3fff700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 0x7f28c4199760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000c00 tx=0x7f28b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c1ffb700 1 -- 172.21.15.96:0/1955550114 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0027070 con 0x7f28c4102300 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c419e470 con 0x7f28c4102300 2024-04-25T20:56:52.480 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c1ffb700 1 -- 172.21.15.96:0/1955550114 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b00041d0 con 0x7f28c4102300 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28c1ffb700 1 -- 172.21.15.96:0/1955550114 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b000f460 con 0x7f28c4102300 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.471+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c41a5d50 con 0x7f28c4102300 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.474+0000 7f28c1ffb700 1 -- 172.21.15.96:0/1955550114 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f28b002a070 con 0x7f28c4102300 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.474+0000 7f28c1ffb700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28b40a3c40 0x7f28b40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28c8941700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28b40a3c40 0x7f28b40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28c1ffb700 1 -- 172.21.15.96:0/1955550114 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f28b00d1340 con 0x7f28c4102300 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28c8941700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28b40a3c40 0x7f28b40a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28b8007900 tx=0x7f28b8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 Processor -- start 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 -- start start 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e63a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e645340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e647c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e05e58dc40 con 0x55e05e63a9d0 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e05e5348e0 con 0x55e05e63a0b0 2024-04-25T20:56:52.481 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28d542eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e05e589c00 con 0x55e05e645880 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.475+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e645340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e645340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50446/0 (socket says 172.21.15.96:50446) 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 -- 172.21.15.96:0/3640275462 learned_addr learned my addr 172.21.15.96:0/3640275462 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c8941700 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e63a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c9142700 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e647c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 -- 172.21.15.96:0/3640275462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 msgr2=0x55e05e647c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e647c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 -- 172.21.15.96:0/3640275462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 msgr2=0x55e05e63a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e63a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 -- 172.21.15.96:0/3640275462 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05e4f86f0 con 0x55e05e63a9d0 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28c3fff700 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e645340 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009350 tx=0x7f28b009d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af03284db48c0f9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28aeffd700 1 -- 172.21.15.96:0/3640275462 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0022470 con 0x55e05e63a9d0 2024-04-25T20:56:52.482 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28aeffd700 1 -- 172.21.15.96:0/3640275462 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b009d750 con 0x55e05e63a9d0 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.476+0000 7f28aeffd700 1 -- 172.21.15.96:0/3640275462 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b009dc10 con 0x55e05e63a9d0 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 -- 172.21.15.96:0/3640275462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 msgr2=0x55e05e645340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e645340 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009350 tx=0x7f28b009d360 comp rx=0 tx=0).stop 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 -- 172.21.15.96:0/3640275462 shutdown_connections 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e647c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e63a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3640275462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e645340 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 -- 172.21.15.96:0/3640275462 >> 172.21.15.96:0/3640275462 conn(0x55e05e567840 msgr2=0x55e05e636580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 -- 172.21.15.96:0/3640275462 shutdown_connections 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 -- 172.21.15.96:0/3640275462 wait complete. 2024-04-25T20:56:52.483 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.477+0000 7f28d542eb80 1 -- start start 2024-04-25T20:56:52.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 Processor -- start 2024-04-25T20:56:52.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e644fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.486 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e6422b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e6427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e05e58dc40 con 0x55e05e63a9d0 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e05e547560 con 0x55e05e63a0b0 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e05e6481b0 con 0x55e05e645880 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28d542eb80 10 client.0 _ll_register_callbacks cb 0x55e05e5268d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e6422b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e6422b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50450/0 (socket says 172.21.15.96:50450) 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 -- 172.21.15.96:0/544383416 learned_addr learned my addr 172.21.15.96:0/544383416 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c8941700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e644fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 -- 172.21.15.96:0/544383416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 msgr2=0x55e05e6427f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e6427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 -- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 msgr2=0x55e05e644fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e644fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 -- 172.21.15.96:0/544383416 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05e4db880 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c8941700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e644fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.478+0000 7f28c3fff700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e6422b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28b0099bb0 tx=0x7f28b009dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad07f6c19795171c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28a3fff700 1 -- 172.21.15.96:0/544383416 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b00bb060 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28a3fff700 1 -- 172.21.15.96:0/544383416 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f28b00a60a0 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b009d750 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28a3fff700 1 -- 172.21.15.96:0/544383416 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28b00a6330 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 10 client.15006 fetch_fsmap learned FSMap version 25 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e05e6528f0 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28a3fff700 1 -- 172.21.15.96:0/544383416 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b009d750 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28a3fff700 1 -- 172.21.15.96:0/544383416 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f28b0086ce0 con 0x55e05e63a9d0 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 10 client.15006 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:56:52.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 10 client.15006 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 20 client.15006 populate_metadata read hostname 'smithi096' 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e654ce0 0x55e05e656150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 4 client.15006 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.479+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e05e59b430 con 0x55e05e654ce0 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.480+0000 7f28c8941700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e654ce0 0x55e05e656150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.480+0000 7f28c8941700 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e654ce0 0x55e05e656150 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aad13a5a716b189 server_cookie=24fd5466d36b825e in_seq=0 out_seq=0 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.480+0000 7f28a3fff700 10 client.15006 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28a3fff700 1 -- 172.21.15.96:0/544383416 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e05e59b430 con 0x55e05e654ce0 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28a3fff700 10 client.15006 handle_command_reply: tid=0 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 client.15006 shutdown 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 msgr2=0x55e05e6422b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e6422b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f28b0099bb0 tx=0x7f28b009dd70 comp rx=0 tx=0).stop 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 shutdown_connections 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e654ce0 0x55e05e656150 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.489 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e645880 0x55e05e6427f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e644fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 --2- 172.21.15.96:0/544383416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a9d0 0x55e05e6422b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 >> 172.21.15.96:0/544383416 conn(0x55e05e567840 msgr2=0x55e05e6361b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28a3fff700 0 client.15006 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 shutdown_connections 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 1 -- 172.21.15.96:0/544383416 wait complete. 2024-04-25T20:56:52.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.481+0000 7f28d542eb80 20 client.15006 trim_cache size 0 max 16384 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 Processor -- start 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 -- start start 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 0x55e05e644af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e636040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e645880 0x55e05e636580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e05e654490 con 0x55e05e63a0b0 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e05e654b60 con 0x55e05e645880 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.501+0000 7f28d542eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e05e655230 con 0x55e05e63a9d0 2024-04-25T20:56:52.503 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 0x55e05e644af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 0x55e05e644af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50466/0 (socket says 172.21.15.96:50466) 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 -- 172.21.15.96:0/1430838720 learned_addr learned my addr 172.21.15.96:0/1430838720 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c3fff700 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e636040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 -- 172.21.15.96:0/1430838720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 msgr2=0x55e05e636040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e636040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 -- 172.21.15.96:0/1430838720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e645880 msgr2=0x55e05e636580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e645880 0x55e05e636580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 -- 172.21.15.96:0/1430838720 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05e63bf20 con 0x55e05e63a0b0 2024-04-25T20:56:52.504 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28c8941700 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 0x55e05e644af0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28b800c430 tx=0x7f28b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f1cc5bfb79ebec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.504 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:52 smithi096 ceph-mon[29288]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:52.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28ae7fc700 1 -- 172.21.15.96:0/1430838720 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b800cdd0 con 0x55e05e63a0b0 2024-04-25T20:56:52.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28ae7fc700 1 -- 172.21.15.96:0/1430838720 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8023cf0 con 0x55e05e63a0b0 2024-04-25T20:56:52.508 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.502+0000 7f28ae7fc700 1 -- 172.21.15.96:0/1430838720 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8026900 con 0x55e05e63a0b0 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 -- 172.21.15.96:0/1430838720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 msgr2=0x55e05e644af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 0x55e05e644af0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f28b800c430 tx=0x7f28b8012000 comp rx=0 tx=0).stop 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 -- 172.21.15.96:0/1430838720 shutdown_connections 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e645880 0x55e05e636580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e63a0b0 0x55e05e644af0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 --2- 172.21.15.96:0/1430838720 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e636040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 -- 172.21.15.96:0/1430838720 >> 172.21.15.96:0/1430838720 conn(0x55e05e567840 msgr2=0x55e05e653c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 -- 172.21.15.96:0/1430838720 shutdown_connections 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 -- 172.21.15.96:0/1430838720 wait complete. 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 -- start start 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.503+0000 7f28d542eb80 1 Processor -- start 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e65cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e65d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.509 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 0x55e05e65d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e05e655fd0 con 0x55e05e645880 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e05e655900 con 0x55e05e63a0b0 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e05e6566a0 con 0x55e05e63a9d0 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28d542eb80 10 client.0 _ll_register_callbacks cb 0x55e05e5268d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 0x55e05e65d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 0x55e05e65d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50472/0 (socket says 172.21.15.96:50472) 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 -- 172.21.15.96:0/3700265485 learned_addr learned my addr 172.21.15.96:0/3700265485 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c8941700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e65cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c3fff700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e65d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 -- 172.21.15.96:0/3700265485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 msgr2=0x55e05e65d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e65d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 -- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 msgr2=0x55e05e65cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:56:52.510 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e65cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.504+0000 7f28c9142700 1 -- 172.21.15.96:0/3700265485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e05e636ac0 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28c9142700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 0x55e05e65d680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005bd0 tx=0x7f28bc00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da4cb10e5909ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28a3fff700 1 -- 172.21.15.96:0/3700265485 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28bc019020 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28a3fff700 1 -- 172.21.15.96:0/3700265485 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f28bc014660 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28b8023cf0 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28a3fff700 1 -- 172.21.15.96:0/3700265485 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28bc014860 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 10 client.15012 fetch_fsmap learned FSMap version 25 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e05e6bc2d0 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28a3fff700 1 -- 172.21.15.96:0/3700265485 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8023cf0 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28a3fff700 1 -- 172.21.15.96:0/3700265485 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f28bc020c10 con 0x55e05e645880 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 10 client.15012 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 10 client.15012 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 20 client.15012 populate_metadata read hostname 'smithi096' 2024-04-25T20:56:52.511 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e6c0f60 0x55e05e6bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 4 client.15012 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.505+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "status"}) -- 0x55e05e5954d0 con 0x55e05e6c0f60 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.506+0000 7f28c8941700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e6c0f60 0x55e05e6bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.506+0000 7f28c8941700 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e6c0f60 0x55e05e6bfc70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f1004f98d51549 server_cookie=1161a92a7fcbf944 in_seq=0 out_seq=0 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.506+0000 7f28a3fff700 10 client.15012 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.506+0000 7f28a3fff700 1 -- 172.21.15.96:0/3700265485 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e05e5954d0 con 0x55e05e6c0f60 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.506+0000 7f28a3fff700 10 client.15012 handle_command_reply: tid=0 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.506+0000 7f28d542eb80 1 client.15012 shutdown 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 msgr2=0x55e05e65d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 0x55e05e65d680 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f28bc005bd0 tx=0x7f28bc00fba0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 shutdown_connections 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e05e645880 0x55e05e65d680 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.512 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e05e63a0b0 0x55e05e65cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e05e6c0f60 0x55e05e6bfc70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 --2- 172.21.15.96:0/3700265485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e05e63a9d0 0x55e05e65d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28a3fff700 0 client.15012 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 >> 172.21.15.96:0/3700265485 conn(0x55e05e567840 msgr2=0x55e05e6407b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 shutdown_connections 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 1 -- 172.21.15.96:0/3700265485 wait complete. 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.507+0000 7f28d542eb80 20 client.15012 trim_cache size 0 max 16384 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T20:56:52.513 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24443, 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 141.113014427, 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 93, 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 159.03261330399999 2024-04-25T20:56:52.514 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28b40a3c40 msgr2=0x7f28b40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28b40a3c40 0x7f28b40a6100 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f28b8007900 tx=0x7f28b8008040 comp rx=0 tx=0).stop 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 msgr2=0x7f28c4199760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 0x7f28c4199760 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000c00 tx=0x7f28b00047b0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 shutdown_connections 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f28b40a3c40 0x7f28b40a6100 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f28c4107990 0x7f28c419daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f28c4101940 0x7f28c4199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 --2- 172.21.15.96:0/1955550114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28c4102300 0x7f28c4199760 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 >> 172.21.15.96:0/1955550114 conn(0x7f28c40788f0 msgr2=0x7f28c406aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 shutdown_connections 2024-04-25T20:56:52.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:52.508+0000 7f28caba5700 1 -- 172.21.15.96:0/1955550114 wait complete. 2024-04-25T20:56:52.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:52 smithi097 ceph-mon[32667]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:52.675 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:52 smithi139 ceph-mon[33027]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:53.201 INFO:tasks.cephadm:Setup complete, yielding 2024-04-25T20:56:53.201 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T20:56:53.212 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:56:53.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-25T20:56:53.454 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:54.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:54 smithi096 ceph-mon[29288]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:54 smithi097 ceph-mon[32667]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:54 smithi139 ceph-mon[33027]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 -- 172.21.15.96:0/257009349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848104640 msgr2=0x7f984810aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 --2- 172.21.15.96:0/257009349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848104640 0x7f984810aef0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f984000b0d0 tx=0x7f984000b3e0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 -- 172.21.15.96:0/257009349 shutdown_connections 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 --2- 172.21.15.96:0/257009349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848104640 0x7f984810aef0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 --2- 172.21.15.96:0/257009349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9848102aa0 0x7f9848102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 --2- 172.21.15.96:0/257009349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9848103ca0 0x7f9848104100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 -- 172.21.15.96:0/257009349 >> 172.21.15.96:0/257009349 conn(0x7f98480fe040 msgr2=0x7f9848100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 -- 172.21.15.96:0/257009349 shutdown_connections 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.117+0000 7f984e62a700 1 -- 172.21.15.96:0/257009349 wait complete. 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 Processor -- start 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 -- start start 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9848102aa0 0x7f9848198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 0x7f98481990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9848104640 0x7f98481a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f984810e590 con 0x7f9848103ca0 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f984810e410 con 0x7f9848104640 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984e62a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f984810e710 con 0x7f9848102aa0 2024-04-25T20:56:55.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 0x7f98481990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 0x7f98481990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:50492/0 (socket says 172.21.15.96:50492) 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 -- 172.21.15.96:0/2486427094 learned_addr learned my addr 172.21.15.96:0/2486427094 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f984cbc7700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9848104640 0x7f98481a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f9847fff700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9848102aa0 0x7f9848198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 -- 172.21.15.96:0/2486427094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9848102aa0 msgr2=0x7f9848198ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9848102aa0 0x7f9848198ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 -- 172.21.15.96:0/2486427094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9848104640 msgr2=0x7f98481a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9848104640 0x7f98481a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 -- 172.21.15.96:0/2486427094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98481a0800 con 0x7f9848103ca0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.118+0000 7f98477fe700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 0x7f98481990e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f983c00b560 tx=0x7f983c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.119+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c015020 con 0x7f9848103ca0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.119+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f983c011ce0 con 0x7f9848103ca0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.119+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98481a0af0 con 0x7f9848103ca0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.119+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983c010d20 con 0x7f9848103ca0 2024-04-25T20:56:55.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.119+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98481a1030 con 0x7f9848103ca0 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.120+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9848069730 con 0x7f9848103ca0 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.121+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f983c018020 con 0x7f9848103ca0 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.121+0000 7f98457fa700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f98300a3c40 0x7f98300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.121+0000 7f9847fff700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f98300a3c40 0x7f98300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.121+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f983c0cfae0 con 0x7f9848103ca0 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.122+0000 7f9847fff700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f98300a3c40 0x7f98300a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9838005fd0 tx=0x7f9838005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:55.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.126+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f983c01f020 con 0x7f9848103ca0 2024-04-25T20:56:55.346 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.344+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f9848062e80 con 0x7f98300a3c40 2024-04-25T20:56:55.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.345+0000 7f98457fa700 1 -- 172.21.15.96:0/2486427094 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f9848062e80 con 0x7f98300a3c40 2024-04-25T20:56:55.347 INFO:teuthology.orchestra.run.smithi096.stdout:Backend: cephadm 2024-04-25T20:56:55.347 INFO:teuthology.orchestra.run.smithi096.stdout:Available: Yes 2024-04-25T20:56:55.347 INFO:teuthology.orchestra.run.smithi096.stdout:Paused: No 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.347+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f98300a3c40 msgr2=0x7f98300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.347+0000 7f984e62a700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f98300a3c40 0x7f98300a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9838005fd0 tx=0x7f9838005dd0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.347+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 msgr2=0x7f98481990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.347+0000 7f984e62a700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 0x7f98481990e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f983c00b560 tx=0x7f983c00b870 comp rx=0 tx=0).stop 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 shutdown_connections 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f98300a3c40 0x7f98300a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9848104640 0x7f98481a0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9848102aa0 0x7f9848198ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 --2- 172.21.15.96:0/2486427094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9848103ca0 0x7f98481990e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 >> 172.21.15.96:0/2486427094 conn(0x7f98480fe040 msgr2=0x7f9848106fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 shutdown_connections 2024-04-25T20:56:55.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:55.348+0000 7f984e62a700 1 -- 172.21.15.96:0/2486427094 wait complete. 2024-04-25T20:56:56.043 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-25T20:56:56.282 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:56.681 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:56 smithi096 ceph-mon[29288]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:56.682 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:56 smithi096 ceph-mon[29288]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:56 smithi097 ceph-mon[32667]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:56 smithi097 ceph-mon[32667]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:56.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:56 smithi139 ceph-mon[33027]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:56:56.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:56 smithi139 ceph-mon[33027]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 -- 172.21.15.96:0/2686882083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103540 msgr2=0x7f041c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 --2- 172.21.15.96:0/2686882083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103540 0x7f041c109df0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f041400b0d0 tx=0x7f041400b3e0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 -- 172.21.15.96:0/2686882083 shutdown_connections 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 --2- 172.21.15.96:0/2686882083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c103540 0x7f041c109df0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 --2- 172.21.15.96:0/2686882083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f041c1019a0 0x7f041c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 --2- 172.21.15.96:0/2686882083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 0x7f041c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.006+0000 7f0420ef6700 1 -- 172.21.15.96:0/2686882083 >> 172.21.15.96:0/2686882083 conn(0x7f041c0fcf40 msgr2=0x7f041c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:58.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 -- 172.21.15.96:0/2686882083 shutdown_connections 2024-04-25T20:56:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 -- 172.21.15.96:0/2686882083 wait complete. 2024-04-25T20:56:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 Processor -- start 2024-04-25T20:56:58.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 -- start start 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 0x7f041c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 0x7f041c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f041c103540 0x7f041c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d440 con 0x7f041c1019a0 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d2c0 con 0x7f041c102ba0 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.007+0000 7f0420ef6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f041c10d5c0 con 0x7f041c103540 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 0x7f041c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 0x7f041c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53000/0 (socket says 172.21.15.96:53000) 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 -- 172.21.15.96:0/4204801218 learned_addr learned my addr 172.21.15.96:0/4204801218 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041ad9d700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f041c103540 0x7f041c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f0419d9b700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 0x7f041c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 -- 172.21.15.96:0/4204801218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f041c103540 msgr2=0x7f041c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f041c103540 0x7f041c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 -- 172.21.15.96:0/4204801218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 msgr2=0x7f041c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 0x7f041c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 -- 172.21.15.96:0/4204801218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c19f630 con 0x7f041c1019a0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f0419d9b700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 0x7f041c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f041a59c700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 0x7f041c1979d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f040c00d9b0 tx=0x7f040c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c017070 con 0x7f041c1019a0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c19f920 con 0x7f041c1019a0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040c009de0 con 0x7f041c1019a0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c00f3c0 con 0x7f041c1019a0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.008+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c19fe10 con 0x7f041c1019a0 2024-04-25T20:56:58.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.009+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f041c069730 con 0x7f041c1019a0 2024-04-25T20:56:58.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.010+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f040c009930 con 0x7f041c1019a0 2024-04-25T20:56:58.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.011+0000 7f040b7fe700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f04040a3c40 0x7f04040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:56:58.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.011+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f040c0cfec0 con 0x7f041c1019a0 2024-04-25T20:56:58.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.011+0000 7f0419d9b700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f04040a3c40 0x7f04040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:56:58.013 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.011+0000 7f0419d9b700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f04040a3c40 0x7f04040a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0410005fd0 tx=0x7f0410005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:56:58.016 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.015+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f040c09c940 con 0x7f041c1019a0 2024-04-25T20:56:58.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.221+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f041c062e80 con 0x7f04040a3c40 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.235+0000 7f040b7fe700 1 -- 172.21.15.96:0/4204801218 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f041c062e80 con 0x7f04040a3c40 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.a smithi096 running (2m) 26s ago 2m 15.8M - 19.0.0-3156-gd69f13d8 da4b004c09e8 49a4d99058f8 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.b smithi097 running (2m) 29s ago 2m 29.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 37504cce1767 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.c smithi139 running (3m) 29s ago 3m 21.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c1a2b34257bd 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.d smithi096 running (3m) 26s ago 3m 14.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 77a0792e48e7 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.e smithi097 running (3m) 29s ago 3m 21.3M - 19.0.0-3156-gd69f13d8 da4b004c09e8 5fd78acc5bdd 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.f smithi139 running (3m) 29s ago 3m 18.4M - 19.0.0-3156-gd69f13d8 da4b004c09e8 c4c3a25e5f5c 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.g smithi096 running (2m) 26s ago 2m 14.5M - 19.0.0-3156-gd69f13d8 da4b004c09e8 2984ac248300 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.h smithi097 running (2m) 29s ago 2m 21.0M - 19.0.0-3156-gd69f13d8 da4b004c09e8 f21fa8adbfc8 2024-04-25T20:56:58.236 INFO:teuthology.orchestra.run.smithi096.stdout:mds.i smithi139 running (2m) 29s ago 2m 21.6M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d14a0dc08e86 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mds.j smithi096 running (3m) 26s ago 3m 15.9M - 19.0.0-3156-gd69f13d8 da4b004c09e8 3ea429a3cc86 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mds.k smithi097 running (3m) 29s ago 3m 13.7M - 19.0.0-3156-gd69f13d8 da4b004c09e8 d627f40e39c3 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mds.l smithi139 running (3m) 29s ago 3m 16.1M - 19.0.0-3156-gd69f13d8 da4b004c09e8 226c852ad8e8 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.x smithi096 *:9283,8765 running (10m) 26s ago 10m 515M - 19.0.0-3156-gd69f13d8 da4b004c09e8 8c60889ad849 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.y smithi097 *:8443,8765 running (8m) 29s ago 8m 451M - 19.0.0-3156-gd69f13d8 da4b004c09e8 cb310225ad67 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mgr.z smithi139 *:8443,8765 running (8m) 29s ago 8m 454M - 19.0.0-3156-gd69f13d8 da4b004c09e8 e25241a14a5d 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mon.a smithi096 running (10m) 26s ago 10m 53.1M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 45ed8be0b8db 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mon.b smithi097 running (8m) 29s ago 8m 39.6M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 6bdc7af8f192 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:mon.c smithi139 running (9m) 29s ago 9m 38.9M 2048M 19.0.0-3156-gd69f13d8 da4b004c09e8 7941e855b680 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 smithi096 running (8m) 26s ago 8m 62.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0383cc1c849b 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 smithi097 running (7m) 29s ago 7m 60.0M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 b57053a83abd 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 smithi139 running (7m) 29s ago 7m 52.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 de77454f9250 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 smithi096 running (7m) 26s ago 7m 51.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 16e7dc028564 2024-04-25T20:56:58.237 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 smithi097 running (6m) 29s ago 6m 46.3M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 f3812fd16936 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 smithi139 running (6m) 29s ago 6m 48.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 3586821e4d90 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 smithi096 running (5m) 26s ago 5m 55.1M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 caaadcdfd983 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 smithi097 running (5m) 29s ago 5m 50.8M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 245cf46af4dc 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 smithi139 running (4m) 29s ago 4m 56.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 4b896d54b25b 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 smithi096 running (4m) 26s ago 4m 52.7M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 50b13a2e3a0a 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 smithi097 running (4m) 29s ago 4m 55.2M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 681efe154313 2024-04-25T20:56:58.238 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 smithi139 running (3m) 29s ago 3m 54.4M 4272M 19.0.0-3156-gd69f13d8 da4b004c09e8 0a69513a6b1d 2024-04-25T20:56:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f04040a3c40 msgr2=0x7f04040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f04040a3c40 0x7f04040a6100 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0410005fd0 tx=0x7f0410005e20 comp rx=0 tx=0).stop 2024-04-25T20:56:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 msgr2=0x7f041c1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:56:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 0x7f041c1979d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f040c00d9b0 tx=0x7f040c00dcc0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 shutdown_connections 2024-04-25T20:56:58.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f04040a3c40 0x7f04040a6100 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f041c103540 0x7f041c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f041c1019a0 0x7f041c1979d0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 --2- 172.21.15.96:0/4204801218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f041c102ba0 0x7f041c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:56:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 >> 172.21.15.96:0/4204801218 conn(0x7f041c0fcf40 msgr2=0x7f041c105ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:56:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 shutdown_connections 2024-04-25T20:56:58.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:56:58.237+0000 7f0420ef6700 1 -- 172.21.15.96:0/4204801218 wait complete. 2024-04-25T20:56:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:58 smithi096 ceph-mon[29288]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:58.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:58 smithi097 ceph-mon[32667]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:58.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:58 smithi139 ceph-mon[33027]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:56:58.954 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-25T20:56:59.190 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:56:59.655 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:56:59 smithi096 ceph-mon[29288]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:59.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:56:59 smithi097 ceph-mon[32667]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:56:59.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:56:59 smithi139 ceph-mon[33027]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:00.621 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:00 smithi096 ceph-mon[29288]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:00.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:00 smithi097 ceph-mon[32667]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 -- 172.21.15.96:0/69553930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e641026d0 msgr2=0x7f1e64109f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 --2- 172.21.15.96:0/69553930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e641026d0 0x7f1e64109f90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c00b0d0 tx=0x7f1e5c00b3e0 comp rx=0 tx=0).stop 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 -- 172.21.15.96:0/69553930 shutdown_connections 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 --2- 172.21.15.96:0/69553930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e641026d0 0x7f1e64109f90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 --2- 172.21.15.96:0/69553930 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e64102ff0 0x7f1e64103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 --2- 172.21.15.96:0/69553930 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e64101d30 0x7f1e64102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 -- 172.21.15.96:0/69553930 >> 172.21.15.96:0/69553930 conn(0x7f1e640fc780 msgr2=0x7f1e640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 -- 172.21.15.96:0/69553930 shutdown_connections 2024-04-25T20:57:00.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.866+0000 7f1e68ede700 1 -- 172.21.15.96:0/69553930 wait complete. 2024-04-25T20:57:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 Processor -- start 2024-04-25T20:57:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 -- start start 2024-04-25T20:57:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e64101d30 0x7f1e641017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e641026d0 0x7f1e6419d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:00.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 0x7f1e6419ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6410d550 con 0x7f1e64102ff0 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6410d3d0 con 0x7f1e641026d0 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e68ede700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6410d6d0 con 0x7f1e64101d30 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 0x7f1e6419ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 0x7f1e6419ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53006/0 (socket says 172.21.15.96:53006) 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e61d9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e641026d0 0x7f1e6419d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 -- 172.21.15.96:0/2758302316 learned_addr learned my addr 172.21.15.96:0/2758302316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e6259c700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e64101d30 0x7f1e641017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 -- 172.21.15.96:0/2758302316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e64101d30 msgr2=0x7f1e641017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e64101d30 0x7f1e641017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 -- 172.21.15.96:0/2758302316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e641026d0 msgr2=0x7f1e6419d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:00.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e641026d0 0x7f1e6419d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 -- 172.21.15.96:0/2758302316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e641a0800 con 0x7f1e64102ff0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.867+0000 7f1e62d9d700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 0x7f1e6419ff30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009fd0 tx=0x7f1e5c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.868+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c01f030 con 0x7f1e64102ff0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.868+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e5c007a80 con 0x7f1e64102ff0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.868+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e641a0a90 con 0x7f1e64102ff0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.868+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c004810 con 0x7f1e64102ff0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.868+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e641a0e90 con 0x7f1e64102ff0 2024-04-25T20:57:00.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.869+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e64069730 con 0x7f1e64102ff0 2024-04-25T20:57:00.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.870+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1e5c003a00 con 0x7f1e64102ff0 2024-04-25T20:57:00.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.870+0000 7f1e537fe700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1e4c0a3c40 0x7f1e4c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:00.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.870+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f1e5c0d11d0 con 0x7f1e64102ff0 2024-04-25T20:57:00.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.870+0000 7f1e6259c700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1e4c0a3c40 0x7f1e4c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:00.872 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.871+0000 7f1e6259c700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1e4c0a3c40 0x7f1e4c0a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:00.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:00 smithi139 ceph-mon[33027]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:00.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:00.876+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e5c09dce0 con 0x7f1e64102ff0 2024-04-25T20:57:01.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.081+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1e64062e80 con 0x7f1e4c0a3c40 2024-04-25T20:57:01.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.087+0000 7f1e537fe700 1 -- 172.21.15.96:0/2758302316 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1e64062e80 con 0x7f1e4c0a3c40 2024-04-25T20:57:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-25T20:57:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:mds.all 12/12 32s ago 3m smithi096=a;smithi096=d;smithi096=g;smithi096=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi139=c;smithi139=f;smithi139=i;smithi139=l;count:12 2024-04-25T20:57:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:mgr 3/3 32s ago 8m smithi096=x;smithi097=y;smithi139=z;count:3 2024-04-25T20:57:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:mon 3/3 32s ago 9m smithi096:172.21.15.96=a;smithi097:172.21.15.97=b;smithi139:172.21.15.139=c;count:3 2024-04-25T20:57:01.088 INFO:teuthology.orchestra.run.smithi096.stdout:osd 12 32s ago - 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1e4c0a3c40 msgr2=0x7f1e4c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1e4c0a3c40 0x7f1e4c0a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54005e50 comp rx=0 tx=0).stop 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 msgr2=0x7f1e6419ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 0x7f1e6419ff30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009fd0 tx=0x7f1e5c004e40 comp rx=0 tx=0).stop 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 shutdown_connections 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f1e4c0a3c40 0x7f1e4c0a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e64102ff0 0x7f1e6419ff30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1e64101d30 0x7f1e641017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 --2- 172.21.15.96:0/2758302316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1e641026d0 0x7f1e6419d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:01.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 >> 172.21.15.96:0/2758302316 conn(0x7f1e640fc780 msgr2=0x7f1e64105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:01.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 shutdown_connections 2024-04-25T20:57:01.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:01.089+0000 7f1e68ede700 1 -- 172.21.15.96:0/2758302316 wait complete. 2024-04-25T20:57:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:01 smithi096 ceph-mon[29288]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:01.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:01 smithi096 ceph-mon[29288]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:01.862 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-25T20:57:01.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:01 smithi097 ceph-mon[32667]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:01.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:01 smithi097 ceph-mon[32667]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:01.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:01 smithi139 ceph-mon[33027]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:01.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:01 smithi139 ceph-mon[33027]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:02.108 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 -- 172.21.15.96:0/1490612192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 msgr2=0x7f704010af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 --2- 172.21.15.96:0/1490612192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f704010af00 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f70300099c0 tx=0x7f7030009cd0 comp rx=0 tx=0).stop 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 -- 172.21.15.96:0/1490612192 shutdown_connections 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 --2- 172.21.15.96:0/1490612192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f704010af00 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 --2- 172.21.15.96:0/1490612192 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 0x7f7040102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 --2- 172.21.15.96:0/1490612192 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7040103cb0 0x7f7040104110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 -- 172.21.15.96:0/1490612192 >> 172.21.15.96:0/1490612192 conn(0x7f70400fe010 msgr2=0x7f7040100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 -- 172.21.15.96:0/1490612192 shutdown_connections 2024-04-25T20:57:03.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.864+0000 7f7048283700 1 -- 172.21.15.96:0/1490612192 wait complete. 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 Processor -- start 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 -- start start 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 0x7f704019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7040103cb0 0x7f704019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f70401a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f704010e640 con 0x7f7040104650 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f704010e4c0 con 0x7f7040103cb0 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7048283700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f704010e7c0 con 0x7f7040102ab0 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f70401a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f70401a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53016/0 (socket says 172.21.15.96:53016) 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 -- 172.21.15.96:0/2823441659 learned_addr learned my addr 172.21.15.96:0/2823441659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:03.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f704581e700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7040103cb0 0x7f704019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f704601f700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 0x7f704019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 -- 172.21.15.96:0/2823441659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 msgr2=0x7f704019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 0x7f704019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 -- 172.21.15.96:0/2823441659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7040103cb0 msgr2=0x7f704019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7040103cb0 0x7f704019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 -- 172.21.15.96:0/2823441659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70401a2ad0 con 0x7f7040104650 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f704601f700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 0x7f704019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.865+0000 7f7046820700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f70401a2430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f70300047b0 tx=0x7f7030005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.866+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7030027070 con 0x7f7040104650 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.866+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70401a2d60 con 0x7f7040104650 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.866+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7030022470 con 0x7f7040104650 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.866+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703000f6c0 con 0x7f7040104650 2024-04-25T20:57:03.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.866+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70401a31b0 con 0x7f7040104650 2024-04-25T20:57:03.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.867+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70401a33e0 con 0x7f7040104650 2024-04-25T20:57:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.868+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f703002a070 con 0x7f7040104650 2024-04-25T20:57:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.868+0000 7f703f7fe700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f70280a3c40 0x7f70280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.868+0000 7f704601f700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f70280a3c40 0x7f70280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.868+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f70300d1340 con 0x7f7040104650 2024-04-25T20:57:03.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.869+0000 7f704601f700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f70280a3c40 0x7f70280a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7040103b10 tx=0x7f7038009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:03.873 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:03.873+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f703005eff0 con 0x7f7040104650 2024-04-25T20:57:04.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.077+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7040076390 con 0x7f70280a3c40 2024-04-25T20:57:04.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.078+0000 7f703f7fe700 1 -- 172.21.15.96:0/2823441659 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7040076390 con 0x7f70280a3c40 2024-04-25T20:57:04.079 INFO:teuthology.orchestra.run.smithi096.stdout:HOST ADDR LABELS STATUS 2024-04-25T20:57:04.080 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 172.21.15.96 2024-04-25T20:57:04.080 INFO:teuthology.orchestra.run.smithi096.stdout:smithi097 172.21.15.97 2024-04-25T20:57:04.080 INFO:teuthology.orchestra.run.smithi096.stdout:smithi139 172.21.15.139 2024-04-25T20:57:04.080 INFO:teuthology.orchestra.run.smithi096.stdout:3 hosts in cluster 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f70280a3c40 msgr2=0x7f70280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f70280a3c40 0x7f70280a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7040103b10 tx=0x7f7038009570 comp rx=0 tx=0).stop 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 msgr2=0x7f70401a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f70401a2430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f70300047b0 tx=0x7f7030005e80 comp rx=0 tx=0).stop 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 shutdown_connections 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f70280a3c40 0x7f70280a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:04.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7040104650 0x7f70401a2430 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:04.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7040102ab0 0x7f704019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:04.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 --2- 172.21.15.96:0/2823441659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7040103cb0 0x7f704019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:04.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 >> 172.21.15.96:0/2823441659 conn(0x7f70400fe010 msgr2=0x7f7040074c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:04.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 shutdown_connections 2024-04-25T20:57:04.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:04.081+0000 7f7048283700 1 -- 172.21.15.96:0/2823441659 wait complete. 2024-04-25T20:57:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:04 smithi097 ceph-mon[32667]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:04 smithi139 ceph-mon[33027]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:04 smithi096 ceph-mon[29288]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:04.810 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-25T20:57:05.051 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:05.455 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:05 smithi096 ceph-mon[29288]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:05.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:05 smithi097 ceph-mon[32667]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:05.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:05 smithi139 ceph-mon[33027]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:06.611 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:06 smithi096 ceph-mon[29288]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:06 smithi097 ceph-mon[32667]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:06 smithi139 ceph-mon[33027]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:06.847 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 -- 172.21.15.96:0/52827555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7990103540 msgr2=0x7f7990109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 --2- 172.21.15.96:0/52827555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7990103540 0x7f7990109df0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f798800b0d0 tx=0x7f798800b3e0 comp rx=0 tx=0).stop 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 -- 172.21.15.96:0/52827555 shutdown_connections 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 --2- 172.21.15.96:0/52827555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7990103540 0x7f7990109df0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 --2- 172.21.15.96:0/52827555 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f79901019a0 0x7f7990101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 --2- 172.21.15.96:0/52827555 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990102ba0 0x7f7990103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 -- 172.21.15.96:0/52827555 >> 172.21.15.96:0/52827555 conn(0x7f79900fcf40 msgr2=0x7f79900ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:06.848 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 -- 172.21.15.96:0/52827555 shutdown_connections 2024-04-25T20:57:06.849 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.846+0000 7f7997f99700 1 -- 172.21.15.96:0/52827555 wait complete. 2024-04-25T20:57:06.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 Processor -- start 2024-04-25T20:57:06.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 -- start start 2024-04-25T20:57:06.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 0x7f7990198dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:06.852 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990102ba0 0x7f7990199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7990103540 0x7f79901a02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f799010d530 con 0x7f79901019a0 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f799010d3b0 con 0x7f7990102ba0 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7997f99700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f799010d6b0 con 0x7f7990103540 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7995d35700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 0x7f7990198dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7995d35700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 0x7f7990198dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53112/0 (socket says 172.21.15.96:53112) 2024-04-25T20:57:06.853 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7995d35700 1 -- 172.21.15.96:0/4114799734 learned_addr learned my addr 172.21.15.96:0/4114799734 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7996536700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7990103540 0x7f79901a02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.847+0000 7f7995534700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990102ba0 0x7f7990199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7995d35700 1 -- 172.21.15.96:0/4114799734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7990103540 msgr2=0x7f79901a02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7995d35700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7990103540 0x7f79901a02f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7995d35700 1 -- 172.21.15.96:0/4114799734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990102ba0 msgr2=0x7f7990199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7995d35700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990102ba0 0x7f7990199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7995d35700 1 -- 172.21.15.96:0/4114799734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79901a0990 con 0x7f79901019a0 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7995d35700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 0x7f7990198dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f798c00b530 tx=0x7f798c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c015020 con 0x7f79901019a0 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79901a0c80 con 0x7f79901019a0 2024-04-25T20:57:06.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798c011ce0 con 0x7f79901019a0 2024-04-25T20:57:06.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c010c60 con 0x7f79901019a0 2024-04-25T20:57:06.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.848+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a11c0 con 0x7f79901019a0 2024-04-25T20:57:06.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.849+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7990069730 con 0x7f79901019a0 2024-04-25T20:57:06.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.850+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f798c018020 con 0x7f79901019a0 2024-04-25T20:57:06.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.851+0000 7f7986ffd700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:06.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.851+0000 7f7995534700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:06.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.851+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f798c0cfff0 con 0x7f79901019a0 2024-04-25T20:57:06.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.851+0000 7f7995534700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7980009a10 tx=0x7f7980009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:06.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:06.855+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f798c01f020 con 0x7f79901019a0 2024-04-25T20:57:07.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.064+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f799006ae30 con 0x7f797c0a3c40 2024-04-25T20:57:07.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.066+0000 7f7986ffd700 1 -- 172.21.15.96:0/4114799734 <== mgr.14152 v2:172.21.15.96:6800/23858312 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f799006ae30 con 0x7f797c0a3c40 2024-04-25T20:57:07.067 INFO:teuthology.orchestra.run.smithi096.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-25T20:57:07.067 INFO:teuthology.orchestra.run.smithi096.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T20:57:07.067 INFO:teuthology.orchestra.run.smithi096.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-25T20:57:07.067 INFO:teuthology.orchestra.run.smithi096.stdout:smithi139 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620500N6400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-25T20:57:07.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 msgr2=0x7f797c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:07.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7980009a10 tx=0x7f7980009190 comp rx=0 tx=0).stop 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 msgr2=0x7f7990198dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 0x7f7990198dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f798c00b530 tx=0x7f798c00b840 comp rx=0 tx=0).stop 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 shutdown_connections 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f797c0a3c40 0x7f797c0a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7990103540 0x7f79901a02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79901019a0 0x7f7990198dd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 --2- 172.21.15.96:0/4114799734 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7990102ba0 0x7f7990199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 >> 172.21.15.96:0/4114799734 conn(0x7f79900fcf40 msgr2=0x7f799006a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 shutdown_connections 2024-04-25T20:57:07.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:07.068+0000 7f7997f99700 1 -- 172.21.15.96:0/4114799734 wait complete. 2024-04-25T20:57:07.779 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-25T20:57:07.790 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:57:07.791 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-25T20:57:08.035 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:08.467 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:08 smithi096 ceph-mon[29288]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:08.467 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:08 smithi096 ceph-mon[29288]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:08 smithi097 ceph-mon[32667]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:08 smithi097 ceph-mon[32667]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:08 smithi139 ceph-mon[33027]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:08 smithi139 ceph-mon[33027]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:09.662 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 -- 172.21.15.96:0/4031151553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d01019e0 msgr2=0x7f44d0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 --2- 172.21.15.96:0/4031151553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d01019e0 0x7f44d0101e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f44c40099c0 tx=0x7f44c4009cd0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 -- 172.21.15.96:0/4031151553 shutdown_connections 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 --2- 172.21.15.96:0/4031151553 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44d0103580 0x7f44d0109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 --2- 172.21.15.96:0/4031151553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d01019e0 0x7f44d0101e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 --2- 172.21.15.96:0/4031151553 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d0102be0 0x7f44d0103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.659+0000 7f44d6aae700 1 -- 172.21.15.96:0/4031151553 >> 172.21.15.96:0/4031151553 conn(0x7f44d00fcfa0 msgr2=0x7f44d00ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 -- 172.21.15.96:0/4031151553 shutdown_connections 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 -- 172.21.15.96:0/4031151553 wait complete. 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 Processor -- start 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 -- start start 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d01019e0 0x7f44d0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 0x7f44d0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.663 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44d0103580 0x7f44d019f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010d4d0 con 0x7f44d0102be0 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010d350 con 0x7f44d01019e0 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44d6aae700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f44d010d650 con 0x7f44d0103580 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44cffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 0x7f44d0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44cffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 0x7f44d0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53128/0 (socket says 172.21.15.96:53128) 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.660+0000 7f44cffff700 1 -- 172.21.15.96:0/752728905 learned_addr learned my addr 172.21.15.96:0/752728905 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44d484a700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d01019e0 0x7f44d0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cffff700 1 -- 172.21.15.96:0/752728905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44d0103580 msgr2=0x7f44d019f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cffff700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44d0103580 0x7f44d019f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cffff700 1 -- 172.21.15.96:0/752728905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d01019e0 msgr2=0x7f44d0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cffff700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d01019e0 0x7f44d0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cffff700 1 -- 172.21.15.96:0/752728905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d019f790 con 0x7f44d0102be0 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cffff700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 0x7f44d0198070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c9e0 tx=0x7f44bc00cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:09.664 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc007ac0 con 0x7f44d0102be0 2024-04-25T20:57:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc007c60 con 0x7f44d0102be0 2024-04-25T20:57:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d019fa80 con 0x7f44d0102be0 2024-04-25T20:57:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc018660 con 0x7f44d0102be0 2024-04-25T20:57:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.661+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d006b360 con 0x7f44d0102be0 2024-04-25T20:57:09.665 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.662+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d0069730 con 0x7f44d0102be0 2024-04-25T20:57:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.663+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f44bc00f450 con 0x7f44d0102be0 2024-04-25T20:57:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.664+0000 7f44cdffb700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44c00a5ed0 0x7f44c00a8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.664+0000 7f44d484a700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44c00a5ed0 0x7f44c00a8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.664+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f44bc0d02a0 con 0x7f44d0102be0 2024-04-25T20:57:09.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.664+0000 7f44d484a700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44c00a5ed0 0x7f44c00a8390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44c4000c00 tx=0x7f44c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:09.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.668+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f44bc014030 con 0x7f44d0102be0 2024-04-25T20:57:09.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.911+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f44d006bf60 con 0x7f44d0102be0 2024-04-25T20:57:09.912 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.912+0000 7f44cdffb700 1 -- 172.21.15.96:0/752728905 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3042 (secure 0 0 0) 0x7f44bc09cd20 con 0x7f44d0102be0 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:e25 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:legacy client fscid: 1 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:Filesystem 'cephfs' (1) 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:fs_name cephfs 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 25 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-25T20:54:31.232491+0000 2024-04-25T20:57:09.913 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-25T20:54:49.357397+0000 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:tableserver 0 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:root 0 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:session_timeout 300 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:session_autoclose 300 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:max_file_size 1099511627776 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:max_xattr_size 65536 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:required_client_features {} 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure 0 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:last_failure_osd_epoch 0 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:max_mds 3 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:in 0,1,2 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:up {0=24443,1=24442,2=24422} 2024-04-25T20:57:09.914 INFO:teuthology.orchestra.run.smithi096.stdout:failed 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:damaged 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:stopped 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:data_pools [3] 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:metadata_pool 2 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:inline_data disabled 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:balancer 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:bal_rank_mask -1 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:standby_count_wanted 0 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:qdb_cluster leader: 24443 members: 24442,24422,24443 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.b{0:24443} state up:active seq 6 addr [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.c{0:24421} state up:standby-replay seq 1 addr [v2:172.21.15.139:6836/57160731,v1:172.21.15.139:6837/57160731] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.i{1:24442} state up:active seq 11 addr [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.f{1:24415} state up:standby-replay seq 1 addr [v2:172.21.15.139:6834/1172572610,v1:172.21.15.139:6835/1172572610] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.915 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.h{2:24422} state up:active seq 12 addr [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.e{2:24398} state up:standby-replay seq 1 addr [v2:172.21.15.97:6834/122204709,v1:172.21.15.97:6835/122204709] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:Standby daemons: 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.96:6834/2040913717,v1:172.21.15.96:6835/2040913717] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.96:6836/2090633506,v1:172.21.15.96:6837/2090633506] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.g{-1:14598} state up:standby seq 1 addr [v2:172.21.15.96:6838/2508705262,v1:172.21.15.96:6839/2508705262] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.96:6840/3884645906,v1:172.21.15.96:6841/3884645906] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.k{-1:24383} state up:standby seq 1 addr [v2:172.21.15.97:6832/2453677033,v1:172.21.15.97:6833/2453677033] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.916 INFO:teuthology.orchestra.run.smithi096.stdout:[mds.l{-1:24394} state up:standby seq 1 addr [v2:172.21.15.139:6832/3174744460,v1:172.21.15.139:6833/3174744460] compat {c=[1],r=[1],i=[1fff]}] 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44c00a5ed0 msgr2=0x7f44c00a8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44c00a5ed0 0x7f44c00a8390 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f44c4000c00 tx=0x7f44c4005fb0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 msgr2=0x7f44d0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 0x7f44d0198070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f44bc00c9e0 tx=0x7f44bc00cda0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 shutdown_connections 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f44c00a5ed0 0x7f44c00a8390 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f44d0103580 0x7f44d019f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f44d01019e0 0x7f44d0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 --2- 172.21.15.96:0/752728905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44d0102be0 0x7f44d0198070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:09.917 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 >> 172.21.15.96:0/752728905 conn(0x7f44d00fcfa0 msgr2=0x7f44d0105f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:09.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 shutdown_connections 2024-04-25T20:57:09.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:09.914+0000 7f44d6aae700 1 -- 172.21.15.96:0/752728905 wait complete. 2024-04-25T20:57:09.918 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:57:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:10 smithi097 ceph-mon[32667]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:10 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/752728905' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T20:57:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:10 smithi139 ceph-mon[33027]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:10.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:10 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/752728905' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T20:57:10.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:10 smithi096 ceph-mon[29288]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:10.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:10 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/752728905' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-25T20:57:10.711 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b713778a-0344-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-25T20:57:10.954 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:12 smithi097 ceph-mon[32667]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:12.614 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:12 smithi096 ceph-mon[29288]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 -- 172.21.15.96:0/781356228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac102b90 msgr2=0x7f6dac102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 --2- 172.21.15.96:0/781356228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac102b90 0x7f6dac102ff0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6da00099c0 tx=0x7f6da0009cd0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 -- 172.21.15.96:0/781356228 shutdown_connections 2024-04-25T20:57:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 --2- 172.21.15.96:0/781356228 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac103530 0x7f6dac109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 --2- 172.21.15.96:0/781356228 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dac101990 0x7f6dac101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 --2- 172.21.15.96:0/781356228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac102b90 0x7f6dac102ff0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 -- 172.21.15.96:0/781356228 >> 172.21.15.96:0/781356228 conn(0x7f6dac0fcf30 msgr2=0x7f6dac0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 -- 172.21.15.96:0/781356228 shutdown_connections 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.617+0000 7f6db25e7700 1 -- 172.21.15.96:0/781356228 wait complete. 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 Processor -- start 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 -- start start 2024-04-25T20:57:12.621 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 0x7f6dac06bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dac102b90 0x7f6dac19fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac103530 0x7f6dac1a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac10d4d0 con 0x7f6dac101990 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac10d350 con 0x7f6dac103530 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db25e7700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dac10d650 con 0x7f6dac102b90 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6dabfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 0x7f6dac06bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6dabfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 0x7f6dac06bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53146/0 (socket says 172.21.15.96:53146) 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6dabfff700 1 -- 172.21.15.96:0/1302637694 learned_addr learned my addr 172.21.15.96:0/1302637694 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6db0b84700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac103530 0x7f6dac1a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.622 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.618+0000 7f6dab7fe700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dac102b90 0x7f6dac19fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6dabfff700 1 -- 172.21.15.96:0/1302637694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dac102b90 msgr2=0x7f6dac19fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6dabfff700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dac102b90 0x7f6dac19fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6dabfff700 1 -- 172.21.15.96:0/1302637694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac103530 msgr2=0x7f6dac1a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6dabfff700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac103530 0x7f6dac1a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6dabfff700 1 -- 172.21.15.96:0/1302637694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dac1a2a80 con 0x7f6dac101990 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6dabfff700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 0x7f6dac06bb10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00c990 tx=0x7f6d9c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:12.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c007b00 con 0x7f6dac101990 2024-04-25T20:57:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d9c007ca0 con 0x7f6dac101990 2024-04-25T20:57:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dac1a2d70 con 0x7f6dac101990 2024-04-25T20:57:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c018660 con 0x7f6dac101990 2024-04-25T20:57:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.619+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dac1a3250 con 0x7f6dac101990 2024-04-25T20:57:12.624 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.620+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6dac069730 con 0x7f6dac101990 2024-04-25T20:57:12.625 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:12 smithi139 ceph-mon[33027]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.621+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6d9c00f450 con 0x7f6dac101990 2024-04-25T20:57:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.622+0000 7f6da97fa700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6d940a3c40 0x7f6d940a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.622+0000 7f6dab7fe700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6d940a3c40 0x7f6d940a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.622+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f6d9c0d0290 con 0x7f6dac101990 2024-04-25T20:57:12.626 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.622+0000 7f6dab7fe700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6d940a3c40 0x7f6d940a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6da00098e0 tx=0x7f6da0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:12.627 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.626+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d9c014030 con 0x7f6dac101990 2024-04-25T20:57:12.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.827+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6dac02e020 con 0x7f6dac101990 2024-04-25T20:57:12.827 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.828+0000 7f6da97fa700 1 -- 172.21.15.96:0/1302637694 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5220 (secure 0 0 0) 0x7f6d9c09cd10 con 0x7f6dac101990 2024-04-25T20:57:12.828 INFO:teuthology.orchestra.run.smithi096.stdout:epoch 93 2024-04-25T20:57:12.828 INFO:teuthology.orchestra.run.smithi096.stdout:fsid b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T20:57:12.828 INFO:teuthology.orchestra.run.smithi096.stdout:created 2024-04-25T20:46:14.467106+0000 2024-04-25T20:57:12.828 INFO:teuthology.orchestra.run.smithi096.stdout:modified 2024-04-25T20:55:10.320396+0000 2024-04-25T20:57:12.828 INFO:teuthology.orchestra.run.smithi096.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-25T20:57:12.828 INFO:teuthology.orchestra.run.smithi096.stdout:crush_version 26 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:full_ratio 0.95 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:backfillfull_ratio 0.9 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:nearfull_ratio 0.85 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:require_min_compat_client luminous 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:min_compat_client luminous 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:require_osd_release squid 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:stretch_mode_enabled false 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 91 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 91 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-25T20:57:12.829 INFO:teuthology.orchestra.run.smithi096.stdout:max_osd 12 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6802/1267246354,v1:172.21.15.96:6803/1267246354] [v2:172.21.15.96:6804/1267246354,v1:172.21.15.96:6805/1267246354] exists,up 8fe35739-7b4a-4c48-980e-75a5c703beaa 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.1 up in weight 1 up_from 13 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/3234593794,v1:172.21.15.97:6801/3234593794] [v2:172.21.15.97:6802/3234593794,v1:172.21.15.97:6803/3234593794] exists,up 2a2a1f79-9b1b-416c-9797-2417119176d4 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.2 up in weight 1 up_from 18 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6800/1811270506,v1:172.21.15.139:6801/1811270506] [v2:172.21.15.139:6802/1811270506,v1:172.21.15.139:6803/1811270506] exists,up 68afacf2-e118-4883-b8b9-051a6add7851 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.3 up in weight 1 up_from 26 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6810/2644602517,v1:172.21.15.96:6811/2644602517] [v2:172.21.15.96:6812/2644602517,v1:172.21.15.96:6813/2644602517] exists,up 9cf6c36f-7417-4b70-961e-a1712eb00fa1 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.4 up in weight 1 up_from 34 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/443367186,v1:172.21.15.97:6809/443367186] [v2:172.21.15.97:6810/443367186,v1:172.21.15.97:6811/443367186] exists,up cd60dbbf-deec-4429-b64c-dc7238e7bb49 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.5 up in weight 1 up_from 39 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6808/2385064558,v1:172.21.15.139:6809/2385064558] [v2:172.21.15.139:6810/2385064558,v1:172.21.15.139:6811/2385064558] exists,up 683e5552-4261-443b-8552-397edba9e004 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6818/262813972,v1:172.21.15.96:6819/262813972] [v2:172.21.15.96:6820/262813972,v1:172.21.15.96:6821/262813972] exists,up 9a351fac-72bd-41d4-97a0-5f8010ab0eb4 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.7 up in weight 1 up_from 52 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/2284407992,v1:172.21.15.97:6817/2284407992] [v2:172.21.15.97:6818/2284407992,v1:172.21.15.97:6819/2284407992] exists,up 949efe04-6338-4968-a214-31dec3068888 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.8 up in weight 1 up_from 61 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6816/3100320732,v1:172.21.15.139:6817/3100320732] [v2:172.21.15.139:6818/3100320732,v1:172.21.15.139:6819/3100320732] exists,up eb49bb16-6c0d-41a7-8319-cd4e96b9a89d 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.9 up in weight 1 up_from 69 up_thru 88 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6826/2687652151,v1:172.21.15.96:6827/2687652151] [v2:172.21.15.96:6828/2687652151,v1:172.21.15.96:6829/2687652151] exists,up ffaf81c6-340a-416a-8c8d-ed179d853403 2024-04-25T20:57:12.830 INFO:teuthology.orchestra.run.smithi096.stdout:osd.10 up in weight 1 up_from 75 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/3642406379,v1:172.21.15.97:6825/3642406379] [v2:172.21.15.97:6826/3642406379,v1:172.21.15.97:6827/3642406379] exists,up 872e6405-0b55-4088-ab33-a8a3f4b7ac3e 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:osd.11 up in weight 1 up_from 83 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.139:6824/1534901897,v1:172.21.15.139:6825/1534901897] [v2:172.21.15.139:6826/1534901897,v1:172.21.15.139:6827/1534901897] exists,up 7a27fcf8-65a4-491e-9f62-8e597415e90c 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.5 [8,11] 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 2.a [2,11] 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.3 [4,9] 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.17 [4,8] 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.24 [4,9] 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:pg_upmap_items 3.33 [5,9] 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1163862190 expires 2024-04-26T20:47:09.141019+0000 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2305167259 expires 2024-04-26T20:47:09.141019+0000 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/1894355014 expires 2024-04-26T20:47:09.141019+0000 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6801/3915899740 expires 2024-04-26T20:46:41.969698+0000 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/1496220452 expires 2024-04-26T20:46:41.969698+0000 2024-04-25T20:57:12.831 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/111790105 expires 2024-04-26T20:46:41.969698+0000 2024-04-25T20:57:12.832 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2748921341 expires 2024-04-26T20:46:41.969698+0000 2024-04-25T20:57:12.832 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/1894355014 expires 2024-04-26T20:47:09.141019+0000 2024-04-25T20:57:12.832 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:0/2265596716 expires 2024-04-26T20:47:09.141019+0000 2024-04-25T20:57:12.832 INFO:teuthology.orchestra.run.smithi096.stdout:blocklist 172.21.15.96:6800/3915899740 expires 2024-04-26T20:46:41.969698+0000 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6d940a3c40 msgr2=0x7f6d940a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6d940a3c40 0x7f6d940a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6da00098e0 tx=0x7f6da0005fd0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 msgr2=0x7f6dac06bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 0x7f6dac06bb10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c00c990 tx=0x7f6d9c00cca0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 shutdown_connections 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f6d940a3c40 0x7f6d940a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dac103530 0x7f6dac1a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dac101990 0x7f6dac06bb10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 --2- 172.21.15.96:0/1302637694 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dac102b90 0x7f6dac19fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 >> 172.21.15.96:0/1302637694 conn(0x7f6dac0fcf30 msgr2=0x7f6dac105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 shutdown_connections 2024-04-25T20:57:12.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:12.830+0000 7f6db25e7700 1 -- 172.21.15.96:0/1302637694 wait complete. 2024-04-25T20:57:13.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:13 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1302637694' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T20:57:13.588 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-25T20:57:13.602 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:57:13.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:13 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1302637694' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T20:57:13.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:13 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1302637694' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-25T20:57:13.843 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:14 smithi097 ceph-mon[32667]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:14 smithi139 ceph-mon[33027]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:14.642 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:14 smithi096 ceph-mon[29288]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:15.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.546+0000 7f73dc8a2700 1 -- 172.21.15.96:0/2780646846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d8101930 msgr2=0x7f73d8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.546+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/2780646846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d8101930 0x7f73d8101db0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009990 tx=0x7f73cc009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 -- 172.21.15.96:0/2780646846 shutdown_connections 2024-04-25T20:57:15.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/2780646846 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d81022f0 0x7f73d810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/2780646846 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 0x7f73d8107d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/2780646846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d8101930 0x7f73d8101db0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 -- 172.21.15.96:0/2780646846 >> 172.21.15.96:0/2780646846 conn(0x7f73d80788b0 msgr2=0x7f73d8078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 -- 172.21.15.96:0/2780646846 shutdown_connections 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 -- 172.21.15.96:0/2780646846 wait complete. 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 Processor -- start 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.547+0000 7f73dc8a2700 1 -- start start 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d8101930 0x7f73d8199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 0x7f73d819a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.550 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 0x7f73d81a5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8111b40 con 0x7f73d81022f0 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f73d81119c0 con 0x7f73d8107980 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f73d8111cc0 con 0x7f73d8101930 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 0x7f73d819a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 0x7f73d819a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:53168/0 (socket says 172.21.15.96:53168) 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 -- 172.21.15.96:0/3539811349 learned_addr learned my addr 172.21.15.96:0/3539811349 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d6d9d700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 0x7f73d81a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 -- 172.21.15.96:0/3539811349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d8101930 msgr2=0x7f73d8199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d8101930 0x7f73d8199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 -- 172.21.15.96:0/3539811349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 msgr2=0x7f73d81a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.551 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 0x7f73d81a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 -- 172.21.15.96:0/3539811349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d81a55a0 con 0x7f73d81022f0 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d6d9d700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 0x7f73d81a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73d5d9b700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 0x7f73d819a520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f73cc000c00 tx=0x7f73cc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc027070 con 0x7f73d81022f0 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73cc0041d0 con 0x7f73d81022f0 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d81a5770 con 0x7f73d81022f0 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00f460 con 0x7f73d81022f0 2024-04-25T20:57:15.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d81a5bc0 con 0x7f73d81022f0 2024-04-25T20:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.548+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d8069730 con 0x7f73d81022f0 2024-04-25T20:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.549+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f73cc02a070 con 0x7f73d81022f0 2024-04-25T20:57:15.553 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.550+0000 7f73c77fe700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:15.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.550+0000 7f73d659c700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:15.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.550+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f73cc0d1340 con 0x7f73d81022f0 2024-04-25T20:57:15.554 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.550+0000 7f73d659c700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73c800a990 tx=0x7f73c8005c80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:15.555 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.554+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f73cc05eff0 con 0x7f73d81022f0 2024-04-25T20:57:15.800 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.799+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73d802e140 con 0x7f73d81022f0 2024-04-25T20:57:15.801 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.800+0000 7f73c77fe700 1 -- 172.21.15.96:0/3539811349 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f73cc013070 con 0x7f73d81022f0 2024-04-25T20:57:15.801 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:15.801 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:54:49.357397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:57:15.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 msgr2=0x7f73c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f73c800a990 tx=0x7f73c8005c80 comp rx=0 tx=0).stop 2024-04-25T20:57:15.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 msgr2=0x7f73d819a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:15.803 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 0x7f73d819a520 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f73cc000c00 tx=0x7f73cc0047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 shutdown_connections 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f73c00a3c40 0x7f73c00a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73d8107980 0x7f73d81a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f73d8101930 0x7f73d8199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 --2- 172.21.15.96:0/3539811349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f73d81022f0 0x7f73d819a520 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 >> 172.21.15.96:0/3539811349 conn(0x7f73d80788b0 msgr2=0x7f73d80ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 shutdown_connections 2024-04-25T20:57:15.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:15.802+0000 7f73dc8a2700 1 -- 172.21.15.96:0/3539811349 wait complete. 2024-04-25T20:57:15.805 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:57:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:16 smithi096 ceph-mon[29288]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:16 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3539811349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:16.605 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:57:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:16 smithi097 ceph-mon[32667]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:16 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3539811349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:16 smithi139 ceph-mon[33027]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:16 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3539811349' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:16.843 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:18.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.469+0000 7f818b552700 1 -- 172.21.15.96:0/1352977834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f818406b1b0 msgr2=0x7f818406b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.469+0000 7f818b552700 1 --2- 172.21.15.96:0/1352977834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f818406b1b0 0x7f818406b630 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f81740099c0 tx=0x7f8174009cd0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.473 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- 172.21.15.96:0/1352977834 shutdown_connections 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 --2- 172.21.15.96:0/1352977834 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f818406bb70 0x7f818410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 --2- 172.21.15.96:0/1352977834 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8184108a50 0x7f8184108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 --2- 172.21.15.96:0/1352977834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f818406b1b0 0x7f818406b630 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- 172.21.15.96:0/1352977834 >> 172.21.15.96:0/1352977834 conn(0x7f8184077990 msgr2=0x7f8184077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- 172.21.15.96:0/1352977834 shutdown_connections 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- 172.21.15.96:0/1352977834 wait complete. 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 Processor -- start 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- start start 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f818406b1b0 0x7f818419b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818406bb70 0x7f818419b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 0x7f818419fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8184112c20 con 0x7f8184108a50 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8184112aa0 con 0x7f818406bb70 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.470+0000 7f818b552700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8184112da0 con 0x7f818406b1b0 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 0x7f818419fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 0x7f818419fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43174/0 (socket says 172.21.15.96:43174) 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f81892ee700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f818406b1b0 0x7f818419b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 -- 172.21.15.96:0/4017189521 learned_addr learned my addr 172.21.15.96:0/4017189521 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8188aed700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818406bb70 0x7f818419b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 -- 172.21.15.96:0/4017189521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f818406b1b0 msgr2=0x7f818419b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f818406b1b0 0x7f818419b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 -- 172.21.15.96:0/4017189521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818406bb70 msgr2=0x7f818419b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818406bb70 0x7f818419b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 -- 172.21.15.96:0/4017189521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81841a0290 con 0x7f8184108a50 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f8189aef700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 0x7f818419fb50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f817c007d70 tx=0x7f817c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c016020 con 0x7f8184108a50 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81841a0580 con 0x7f8184108a50 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f817c00ac90 con 0x7f8184108a50 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f817c0115e0 con 0x7f8184108a50 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.471+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81841a7e00 con 0x7f8184108a50 2024-04-25T20:57:18.476 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.472+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8184069730 con 0x7f8184108a50 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.473+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f817c019020 con 0x7f8184108a50 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.474+0000 7f817a7fc700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f81700a3c40 0x7f81700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.474+0000 7f81892ee700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f81700a3c40 0x7f81700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.474+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f817c0d0780 con 0x7f8184108a50 2024-04-25T20:57:18.477 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.474+0000 7f81892ee700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f81700a3c40 0x7f81700a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8180005950 tx=0x7f818000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:18.478 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.478+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f817c020020 con 0x7f8184108a50 2024-04-25T20:57:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:18 smithi097 ceph-mon[32667]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:18 smithi139 ceph-mon[33027]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.723+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81841a8030 con 0x7f8184108a50 2024-04-25T20:57:18.724 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:18 smithi096 ceph-mon[29288]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:18.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.724+0000 7f817a7fc700 1 -- 172.21.15.96:0/4017189521 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11859 (secure 0 0 0) 0x7f817c09d200 con 0x7f8184108a50 2024-04-25T20:57:18.725 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:18.725 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:54:49.357397+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:57:18.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f81700a3c40 msgr2=0x7f81700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f81700a3c40 0x7f81700a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8180005950 tx=0x7f818000b410 comp rx=0 tx=0).stop 2024-04-25T20:57:18.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 msgr2=0x7f818419fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:18.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 0x7f818419fb50 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f817c007d70 tx=0x7f817c00a340 comp rx=0 tx=0).stop 2024-04-25T20:57:18.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 shutdown_connections 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f81700a3c40 0x7f81700a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8184108a50 0x7f818419fb50 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f818406b1b0 0x7f818419b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 --2- 172.21.15.96:0/4017189521 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f818406bb70 0x7f818419b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 >> 172.21.15.96:0/4017189521 conn(0x7f8184077990 msgr2=0x7f818410dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 shutdown_connections 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:18.726+0000 7f818b552700 1 -- 172.21.15.96:0/4017189521 wait complete. 2024-04-25T20:57:18.728 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 25 2024-04-25T20:57:19.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:19 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4017189521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:19.589 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:57:19.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:19 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4017189521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:19.831 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:19.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:19 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4017189521' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:20 smithi139 ceph-mon[33027]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:20.647 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:20 smithi096 ceph-mon[29288]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:20.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:20 smithi097 ceph-mon[32667]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.459+0000 7f4345366700 1 -- 172.21.15.96:0/2049484613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43401026e0 msgr2=0x7f434010e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.459+0000 7f4345366700 1 --2- 172.21.15.96:0/2049484613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43401026e0 0x7f434010e770 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f433800b0a0 tx=0x7f433800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 -- 172.21.15.96:0/2049484613 shutdown_connections 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 --2- 172.21.15.96:0/2049484613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43401026e0 0x7f434010e770 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 --2- 172.21.15.96:0/2049484613 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4340106990 0x7f4340106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 --2- 172.21.15.96:0/2049484613 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4340101d20 0x7f43401021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 -- 172.21.15.96:0/2049484613 >> 172.21.15.96:0/2049484613 conn(0x7f43400788f0 msgr2=0x7f4340078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 -- 172.21.15.96:0/2049484613 shutdown_connections 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 -- 172.21.15.96:0/2049484613 wait complete. 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 Processor -- start 2024-04-25T20:57:21.463 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.460+0000 7f4345366700 1 -- start start 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 0x7f4340199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43401026e0 0x7f434019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4340106990 0x7f43401a4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4340111b70 con 0x7f4340101d20 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f43401119f0 con 0x7f43401026e0 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4340111cf0 con 0x7f4340106990 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 0x7f4340199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 0x7f4340199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43194/0 (socket says 172.21.15.96:43194) 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 -- 172.21.15.96:0/2207547569 learned_addr learned my addr 172.21.15.96:0/2207547569 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433f7fe700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4340106990 0x7f43401a4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433e7fc700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43401026e0 0x7f434019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 -- 172.21.15.96:0/2207547569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4340106990 msgr2=0x7f43401a4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4340106990 0x7f43401a4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 -- 172.21.15.96:0/2207547569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43401026e0 msgr2=0x7f434019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43401026e0 0x7f434019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 -- 172.21.15.96:0/2207547569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43401a5490 con 0x7f4340101d20 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f433effd700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 0x7f4340199ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f433000c9b0 tx=0x7f433000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330007b00 con 0x7f4340101d20 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43401a5660 con 0x7f4340101d20 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4330007ca0 con 0x7f4340101d20 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4330018730 con 0x7f4340101d20 2024-04-25T20:57:21.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.461+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43401a5bc0 con 0x7f4340101d20 2024-04-25T20:57:21.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.462+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4340103080 con 0x7f4340101d20 2024-04-25T20:57:21.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.464+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f433000f450 con 0x7f4340101d20 2024-04-25T20:57:21.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.465+0000 7f4327fff700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f43280a3c40 0x7f43280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:21.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.465+0000 7f433e7fc700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f43280a3c40 0x7f43280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:21.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.465+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f43300d06f0 con 0x7f4340101d20 2024-04-25T20:57:21.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.465+0000 7f433e7fc700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f43280a3c40 0x7f43280a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:21.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.468+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4330014030 con 0x7f4340101d20 2024-04-25T20:57:21.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.668+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f434002df80 con 0x7f4340101d20 2024-04-25T20:57:21.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.670+0000 7f4327fff700 1 -- 172.21.15.96:0/2207547569 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22523 (secure 0 0 0) 0x7f433009d170 con 0x7f4340101d20 2024-04-25T20:57:21.671 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:21.671 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":93,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:55:10.320396+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f43280a3c40 msgr2=0x7f43280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f43280a3c40 0x7f43280a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4334006fd0 tx=0x7f4334008040 comp rx=0 tx=0).stop 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 msgr2=0x7f4340199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 0x7f4340199ed0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f433000c9b0 tx=0x7f433000cd70 comp rx=0 tx=0).stop 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 shutdown_connections 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f43280a3c40 0x7f43280a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4340106990 0x7f43401a4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4340101d20 0x7f4340199ed0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 --2- 172.21.15.96:0/2207547569 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43401026e0 0x7f434019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 >> 172.21.15.96:0/2207547569 conn(0x7f43400788f0 msgr2=0x7f43400ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 shutdown_connections 2024-04-25T20:57:21.673 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:21.672+0000 7f4345366700 1 -- 172.21.15.96:0/2207547569 wait complete. 2024-04-25T20:57:22.397 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:22 smithi096 ceph-mon[29288]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:22.397 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:22 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2207547569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:22.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T20:54:31.232491+0000', 'modified': '2024-04-25T20:54:49.357397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24443, 'mds_1': 24442, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24398': {'gid': 24398, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/122204709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 122204709}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 122204709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6835/1172572610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1172572610}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 1172572610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/57160731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 57160731}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 57160731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.97:6837/16128615', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 16128615}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 16128615}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.139:6839/2573809872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 2573809872}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 2573809872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.97:6839/3461334961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3461334961}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3461334961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24443, 'qdb_cluster': [24442, 24422, 24443]} 2024-04-25T20:57:22.425 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T20:57:22.425 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T20:57:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:22 smithi139 ceph-mon[33027]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:22.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:22 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2207547569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:22.664 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:22.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:22 smithi097 ceph-mon[32667]: pgmap v389: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:22.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:22 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2207547569' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:24.413 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 -- 172.21.15.96:0/136532965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c106940 msgr2=0x7f292c106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 --2- 172.21.15.96:0/136532965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c106940 0x7f292c106d20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2920009990 tx=0x7f2920009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 -- 172.21.15.96:0/136532965 shutdown_connections 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 --2- 172.21.15.96:0/136532965 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f292c1025e0 0x7f292c10e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 --2- 172.21.15.96:0/136532965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c106940 0x7f292c106d20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 --2- 172.21.15.96:0/136532965 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f292c101c20 0x7f292c1020a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 -- 172.21.15.96:0/136532965 >> 172.21.15.96:0/136532965 conn(0x7f292c078830 msgr2=0x7f292c078c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 -- 172.21.15.96:0/136532965 shutdown_connections 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.410+0000 7f2932c56700 1 -- 172.21.15.96:0/136532965 wait complete. 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 Processor -- start 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 -- start start 2024-04-25T20:57:24.414 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 0x7f292c10f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f292c1025e0 0x7f292c10f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f292c106940 0x7f292c10fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f292c112c70 con 0x7f292c101c20 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f292c112af0 con 0x7f292c106940 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f2932c56700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f292c112df0 con 0x7f292c1025e0 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f29309f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 0x7f292c10f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f29309f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 0x7f292c10f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43220/0 (socket says 172.21.15.96:43220) 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f29309f2700 1 -- 172.21.15.96:0/544307383 learned_addr learned my addr 172.21.15.96:0/544307383 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.411+0000 7f292bfff700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f292c1025e0 0x7f292c10f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29311f3700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f292c106940 0x7f292c10fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29309f2700 1 -- 172.21.15.96:0/544307383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f292c1025e0 msgr2=0x7f292c10f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29309f2700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f292c1025e0 0x7f292c10f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.415 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29309f2700 1 -- 172.21.15.96:0/544307383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f292c106940 msgr2=0x7f292c10fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29309f2700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f292c106940 0x7f292c10fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29309f2700 1 -- 172.21.15.96:0/544307383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f292c1a7660 con 0x7f292c101c20 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f29309f2700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 0x7f292c10f0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2920000c00 tx=0x7f29200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f2929ffb700 1 -- 172.21.15.96:0/544307383 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2920027070 con 0x7f292c101c20 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f292c1a78f0 con 0x7f292c101c20 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f2929ffb700 1 -- 172.21.15.96:0/544307383 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29200040e0 con 0x7f292c101c20 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f2929ffb700 1 -- 172.21.15.96:0/544307383 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f292000f460 con 0x7f292c101c20 2024-04-25T20:57:24.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.412+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f292c1a7d40 con 0x7f292c101c20 2024-04-25T20:57:24.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.414+0000 7f2929ffb700 1 -- 172.21.15.96:0/544307383 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f292002a070 con 0x7f292c101c20 2024-04-25T20:57:24.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.415+0000 7f2929ffb700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f291c0a3c40 0x7f291c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.415+0000 7f292bfff700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f291c0a3c40 0x7f291c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.415+0000 7f2929ffb700 1 -- 172.21.15.96:0/544307383 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f29200d1140 con 0x7f292c101c20 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.415+0000 7f292bfff700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f291c0a3c40 0x7f291c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f292c078ee0 tx=0x7f291800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 Processor -- start 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 -- start start 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360e1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 0x55e0360e6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e6580 0x55e0360e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e036033ce0 con 0x55e0360e20f0 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e035fda910 con 0x55e0360e17d0 2024-04-25T20:57:24.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e03602fca0 con 0x55e0360e6580 2024-04-25T20:57:24.418 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:24 smithi096 ceph-mon[29288]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f292bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 0x55e0360e6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f292bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 0x55e0360e6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43230/0 (socket says 172.21.15.96:43230) 2024-04-25T20:57:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f292bfff700 1 -- 172.21.15.96:0/1774464475 learned_addr learned my addr 172.21.15.96:0/1774464475 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:24.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f29311f3700 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e6580 0x55e0360e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.416+0000 7f29309f2700 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360e1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f292bfff700 1 -- 172.21.15.96:0/1774464475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e6580 msgr2=0x55e0360e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f292bfff700 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e6580 0x55e0360e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f292bfff700 1 -- 172.21.15.96:0/1774464475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 msgr2=0x55e0360e1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f292bfff700 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360e1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f292bfff700 1 -- 172.21.15.96:0/1774464475 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e035f81880 con 0x55e0360e20f0 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f292bfff700 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 0x55e0360e6040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f291800fd70 tx=0x7f2918012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e3e0918dca1c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f2916ffd700 1 -- 172.21.15.96:0/1774464475 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2918016070 con 0x55e0360e20f0 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f2916ffd700 1 -- 172.21.15.96:0/1774464475 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2918010d70 con 0x55e0360e20f0 2024-04-25T20:57:24.424 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.417+0000 7f2916ffd700 1 -- 172.21.15.96:0/1774464475 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f291801bd60 con 0x55e0360e20f0 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 -- 172.21.15.96:0/1774464475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 msgr2=0x55e0360e6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 0x55e0360e6040 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f291800fd70 tx=0x7f2918012000 comp rx=0 tx=0).stop 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 -- 172.21.15.96:0/1774464475 shutdown_connections 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e6580 0x55e0360e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360e1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/1774464475 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e20f0 0x55e0360e6040 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 -- 172.21.15.96:0/1774464475 >> 172.21.15.96:0/1774464475 conn(0x55e03600e640 msgr2=0x55e0360dc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 -- 172.21.15.96:0/1774464475 shutdown_connections 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 -- 172.21.15.96:0/1774464475 wait complete. 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 -- start start 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.418+0000 7f293d4dfb80 1 Processor -- start 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e0360edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e0360eadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.425 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0360ee610 con 0x55e0360e6580 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e035fd2b80 con 0x55e0360e20f0 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e0360ee790 con 0x55e0360e17d0 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f293d4dfb80 10 client.0 _ll_register_callbacks cb 0x55e035fcc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360eb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43238/0 (socket says 172.21.15.96:43238) 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 -- 172.21.15.96:0/3913538879 learned_addr learned my addr 172.21.15.96:0/3913538879 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29309f2700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e0360edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f292bfff700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e0360eadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.426 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 -- 172.21.15.96:0/3913538879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 msgr2=0x55e0360edab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e0360edab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 -- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 msgr2=0x55e0360eadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e0360eadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 -- 172.21.15.96:0/3913538879 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e035feb220 con 0x55e0360e6580 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29309f2700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e0360edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f292bfff700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e0360eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f29311f3700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360eb2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f292400be50 tx=0x7f292400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aacc4509e6a269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.419+0000 7f290bfff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f290bfff700 1 -- 172.21.15.96:0/3913538879 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924007ad0 con 0x55e0360e6580 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f290bfff700 1 -- 172.21.15.96:0/3913538879 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f2924007c70 con 0x55e0360e6580 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2918010d70 con 0x55e0360e6580 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f290bfff700 1 -- 172.21.15.96:0/3913538879 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2924023430 con 0x55e0360e6580 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 10 client.15084 fetch_fsmap learned FSMap version 25 2024-04-25T20:57:24.427 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0360f8f40 con 0x55e0360e6580 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f290bfff700 1 -- 172.21.15.96:0/3913538879 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2918010d70 con 0x55e0360e6580 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f290bfff700 1 -- 172.21.15.96:0/3913538879 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2924020030 con 0x55e0360e6580 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 10 client.15084 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 20 client.15084 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e0360fb610 0x55e0360fca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 4 client.15084 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.420+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0360414d0 con 0x55e0360fb610 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.421+0000 7f29309f2700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e0360fb610 0x55e0360fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.421+0000 7f29309f2700 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e0360fb610 0x55e0360fca80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6145ea1b650328d3 server_cookie=fb29de4686dc27a5 in_seq=0 out_seq=0 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.421+0000 7f290bfff700 10 client.15084 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f290bfff700 1 -- 172.21.15.96:0/3913538879 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e0360414d0 con 0x55e0360fb610 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f290bfff700 10 client.15084 handle_command_reply: tid=0 2024-04-25T20:57:24.428 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 client.15084 shutdown 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 msgr2=0x55e0360eb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360eb2f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f292400be50 tx=0x7f292400bf30 comp rx=0 tx=0).stop 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 shutdown_connections 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e0360fb610 0x55e0360fca80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360eb2f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e0360edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3913538879 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e0360eadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 >> 172.21.15.96:0/3913538879 conn(0x55e03600e640 msgr2=0x55e0360e79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f290bfff700 0 client.15084 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 shutdown_connections 2024-04-25T20:57:24.429 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3913538879 wait complete. 2024-04-25T20:57:24.430 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.422+0000 7f293d4dfb80 20 client.15084 trim_cache size 0 max 16384 2024-04-25T20:57:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.442+0000 7f293d4dfb80 1 Processor -- start 2024-04-25T20:57:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.442+0000 7f293d4dfb80 1 -- start start 2024-04-25T20:57:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360eb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e20f0 0x55e0360eccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360e80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0360fadc0 con 0x55e0360e6580 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e0360fb490 con 0x55e0360e17d0 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e0360fbb60 con 0x55e0360e20f0 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360e80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360e80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43254/0 (socket says 172.21.15.96:43254) 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 -- 172.21.15.96:0/731383395 learned_addr learned my addr 172.21.15.96:0/731383395 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 -- 172.21.15.96:0/731383395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e20f0 msgr2=0x55e0360eccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e20f0 0x55e0360eccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 -- 172.21.15.96:0/731383395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 msgr2=0x55e0360eb340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360eb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 -- 172.21.15.96:0/731383395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0360ddfb0 con 0x55e0360e6580 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29311f3700 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360e80d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2924004c20 tx=0x7f2924009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4416fac370f91bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29167fc700 1 -- 172.21.15.96:0/731383395 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924018950 con 0x55e0360e6580 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29167fc700 1 -- 172.21.15.96:0/731383395 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29240078d0 con 0x55e0360e6580 2024-04-25T20:57:24.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.443+0000 7f29167fc700 1 -- 172.21.15.96:0/731383395 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924023aa0 con 0x55e0360e6580 2024-04-25T20:57:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 -- 172.21.15.96:0/731383395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 msgr2=0x55e0360e80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360e80d0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2924004c20 tx=0x7f2924009520 comp rx=0 tx=0).stop 2024-04-25T20:57:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 -- 172.21.15.96:0/731383395 shutdown_connections 2024-04-25T20:57:24.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360e80d0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e17d0 0x55e0360eb340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/731383395 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e20f0 0x55e0360eccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 -- 172.21.15.96:0/731383395 >> 172.21.15.96:0/731383395 conn(0x55e03600e640 msgr2=0x55e0360ec5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 -- 172.21.15.96:0/731383395 shutdown_connections 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 -- 172.21.15.96:0/731383395 wait complete. 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 -- start start 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.444+0000 7f293d4dfb80 1 Processor -- start 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e036105f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e036106440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0360fc900 con 0x55e0360e6580 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e0360fc230 con 0x55e0360e20f0 2024-04-25T20:57:24.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e0360fcfd0 con 0x55e0360e17d0 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 10 client.0 _ll_register_callbacks cb 0x55e035fcc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:43258/0 (socket says 172.21.15.96:43258) 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 -- 172.21.15.96:0/3536218203 learned_addr learned my addr 172.21.15.96:0/3536218203 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29309f2700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e036105f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f292bfff700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e036106440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 -- 172.21.15.96:0/3536218203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 msgr2=0x55e036105f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e036105f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 -- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 msgr2=0x55e036106440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e036106440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 -- 172.21.15.96:0/3536218203 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0360ff8c0 con 0x55e0360e6580 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f292bfff700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e036106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f29311f3700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360ff080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2924007d30 tx=0x7f292400bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10bf210aa551ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:24.451 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f290bfff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f290bfff700 1 -- 172.21.15.96:0/3536218203 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2924019b70 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f290bfff700 1 -- 172.21.15.96:0/3536218203 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f2924019d10 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29240078d0 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.445+0000 7f290bfff700 1 -- 172.21.15.96:0/3536218203 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29240235d0 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 10 client.15090 fetch_fsmap learned FSMap version 25 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e036163450 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f290bfff700 1 -- 172.21.15.96:0/3536218203 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29240078d0 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f290bfff700 1 -- 172.21.15.96:0/3536218203 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2924020040 con 0x55e0360e6580 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 10 client.15090 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 20 client.15090 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e036167d60 0x55e036166a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 4 client.15090 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "status"}) -- 0x55e03603b570 con 0x55e036167d60 2024-04-25T20:57:24.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.446+0000 7f29309f2700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e036167d60 0x55e036166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f290bfff700 10 client.15090 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f29309f2700 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e036167d60 0x55e036166a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9005dab9a0a08565 server_cookie=adf44e26cc9f4be6 in_seq=0 out_seq=0 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f290bfff700 1 -- 172.21.15.96:0/3536218203 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e03603b570 con 0x55e036167d60 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f290bfff700 10 client.15090 handle_command_reply: tid=0 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 client.15090 shutdown 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 msgr2=0x55e0360ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360ff080 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2924007d30 tx=0x7f292400bf30 comp rx=0 tx=0).stop 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 shutdown_connections 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55e036167d60 0x55e036166a70 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0360e6580 0x55e0360ff080 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e0360e17d0 0x55e036105f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 --2- 172.21.15.96:0/3536218203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e0360e20f0 0x55e036106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 >> 172.21.15.96:0/3536218203 conn(0x55e03600e640 msgr2=0x55e0360e0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f290bfff700 0 client.15090 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 shutdown_connections 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 1 -- 172.21.15.96:0/3536218203 wait complete. 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.447+0000 7f293d4dfb80 20 client.15090 trim_cache size 0 max 16384 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24422, 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:57:24.454 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 160.12221613400001, 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 93, 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 203.78426692900001 2024-04-25T20:57:24.455 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f291c0a3c40 msgr2=0x7f291c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f291c0a3c40 0x7f291c0a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f292c078ee0 tx=0x7f291800a300 comp rx=0 tx=0).stop 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 msgr2=0x7f292c10f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 0x7f292c10f0e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2920000c00 tx=0x7f29200047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 shutdown_connections 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f291c0a3c40 0x7f291c0a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f292c106940 0x7f292c10fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f292c101c20 0x7f292c10f0e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 --2- 172.21.15.96:0/544307383 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f292c1025e0 0x7f292c10f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 >> 172.21.15.96:0/544307383 conn(0x7f292c078830 msgr2=0x7f292c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 shutdown_connections 2024-04-25T20:57:24.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:24.448+0000 7f2932c56700 1 -- 172.21.15.96:0/544307383 wait complete. 2024-04-25T20:57:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:24 smithi139 ceph-mon[33027]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:24.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:24 smithi097 ceph-mon[32667]: pgmap v390: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T20:57:25.142 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T20:57:25.385 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:26.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:26 smithi096 ceph-mon[29288]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:26 smithi139 ceph-mon[33027]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:26 smithi097 ceph-mon[32667]: pgmap v391: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-04-25T20:57:27.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 -- 172.21.15.96:0/1739486858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938103450 msgr2=0x7f793810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1739486858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938103450 0x7f793810f8b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f793000b0a0 tx=0x7f793000b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 -- 172.21.15.96:0/1739486858 shutdown_connections 2024-04-25T20:57:27.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1739486858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938103450 0x7f793810f8b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1739486858 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7938108a50 0x7f7938108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1739486858 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938102a90 0x7f7938102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 -- 172.21.15.96:0/1739486858 >> 172.21.15.96:0/1739486858 conn(0x7f7938079ba0 msgr2=0x7f7938079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 -- 172.21.15.96:0/1739486858 shutdown_connections 2024-04-25T20:57:27.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 -- 172.21.15.96:0/1739486858 wait complete. 2024-04-25T20:57:27.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 Processor -- start 2024-04-25T20:57:27.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.130+0000 7f793ee5d700 1 -- start start 2024-04-25T20:57:27.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7938102a90 0x7f7938199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938103450 0x7f79381996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 0x7f793819d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7938112c80 con 0x7f7938108a50 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7938112b00 con 0x7f7938102a90 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7938112e00 con 0x7f7938103450 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 0x7f793819d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 0x7f793819d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59696/0 (socket says 172.21.15.96:59696) 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 -- 172.21.15.96:0/1406238578 learned_addr learned my addr 172.21.15.96:0/1406238578 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f792ffff700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938103450 0x7f79381996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 -- 172.21.15.96:0/1406238578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938103450 msgr2=0x7f79381996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938103450 0x7f79381996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 -- 172.21.15.96:0/1406238578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7938102a90 msgr2=0x7f7938199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7938102a90 0x7f7938199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 -- 172.21.15.96:0/1406238578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793819e130 con 0x7f7938108a50 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f792ffff700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938103450 0x7f79381996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793d3fa700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 0x7f793819d9f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79300062a0 tx=0x7f7930004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f792dffb700 1 -- 172.21.15.96:0/1406238578 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7930021070 con 0x7f7938108a50 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f792dffb700 1 -- 172.21.15.96:0/1406238578 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7930007d00 con 0x7f7938108a50 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793819e3c0 con 0x7f7938108a50 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.131+0000 7f792dffb700 1 -- 172.21.15.96:0/1406238578 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793001d450 con 0x7f7938108a50 2024-04-25T20:57:27.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.132+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79381a5ca0 con 0x7f7938108a50 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.134+0000 7f792dffb700 1 -- 172.21.15.96:0/1406238578 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7930004020 con 0x7f7938108a50 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.134+0000 7f792dffb700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f79200a3c40 0x7f79200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.134+0000 7f793cbf9700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f79200a3c40 0x7f79200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.134+0000 7f792dffb700 1 -- 172.21.15.96:0/1406238578 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f79300d15d0 con 0x7f7938108a50 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f793cbf9700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f79200a3c40 0x7f79200a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79340097b0 tx=0x7f7934006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 Processor -- start 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 -- start start 2024-04-25T20:57:27.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6840b0 0x559dcb684490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6849d0 0x559dcb68f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb691c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dcb5d7c40 con 0x559dcb68f880 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dcb57e8e0 con 0x559dcb6849d0 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.135+0000 7f79496e6b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559dcb5d3c00 con 0x559dcb6840b0 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb691c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb691c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59710/0 (socket says 172.21.15.96:59710) 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 -- 172.21.15.96:0/2897885924 learned_addr learned my addr 172.21.15.96:0/2897885924 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793cbf9700 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6840b0 0x559dcb684490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 -- 172.21.15.96:0/2897885924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6840b0 msgr2=0x559dcb684490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6840b0 0x559dcb684490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f792ffff700 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6849d0 0x559dcb68f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 -- 172.21.15.96:0/2897885924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6849d0 msgr2=0x559dcb68f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6849d0 0x559dcb68f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 -- 172.21.15.96:0/2897885924 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcb5426f0 con 0x559dcb68f880 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f793d3fa700 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb691c70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7930099de0 tx=0x7f79300043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cd3e4bc0353bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f791effd700 1 -- 172.21.15.96:0/2897885924 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793002a070 con 0x559dcb68f880 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f791effd700 1 -- 172.21.15.96:0/2897885924 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793009d980 con 0x559dcb68f880 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.136+0000 7f791effd700 1 -- 172.21.15.96:0/2897885924 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793009dbe0 con 0x559dcb68f880 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 -- 172.21.15.96:0/2897885924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 msgr2=0x559dcb691c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb691c70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7930099de0 tx=0x7f79300043c0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 -- 172.21.15.96:0/2897885924 shutdown_connections 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb691c70 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6840b0 0x559dcb684490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2897885924 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6849d0 0x559dcb68f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 -- 172.21.15.96:0/2897885924 >> 172.21.15.96:0/2897885924 conn(0x559dcb5b1840 msgr2=0x559dcb680580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 -- 172.21.15.96:0/2897885924 shutdown_connections 2024-04-25T20:57:27.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 -- 172.21.15.96:0/2897885924 wait complete. 2024-04-25T20:57:27.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 -- start start 2024-04-25T20:57:27.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.137+0000 7f79496e6b80 1 Processor -- start 2024-04-25T20:57:27.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb68eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb68c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb68c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6965c0 con 0x559dcb6849d0 2024-04-25T20:57:27.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dcb591560 con 0x559dcb6840b0 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559dcb696740 con 0x559dcb68f880 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f79496e6b80 10 client.0 _ll_register_callbacks cb 0x559dcb5708d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb68c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb68c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59720/0 (socket says 172.21.15.96:59720) 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 -- 172.21.15.96:0/3672190732 learned_addr learned my addr 172.21.15.96:0/3672190732 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f793cbf9700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb68eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f793d3fa700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb68c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 -- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 msgr2=0x559dcb68c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb68c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 -- 172.21.15.96:0/3672190732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 msgr2=0x559dcb68eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb68eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 -- 172.21.15.96:0/3672190732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcb525880 con 0x559dcb6849d0 2024-04-25T20:57:27.145 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f793d3fa700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb68c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f792ffff700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb68c2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f792800f3e0 tx=0x7f792800f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1d7167a58afa076 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.138+0000 7f7917fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f7917fff700 1 -- 172.21.15.96:0/3672190732 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928004a30 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f7917fff700 1 -- 172.21.15.96:0/3672190732 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f7928004bd0 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f793009d980 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f7917fff700 1 -- 172.21.15.96:0/3672190732 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7928004dd0 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 10 client.15102 fetch_fsmap learned FSMap version 25 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559dcb6a0dc0 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f7917fff700 1 -- 172.21.15.96:0/3672190732 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f793009d980 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f7917fff700 1 -- 172.21.15.96:0/3672190732 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7928011c10 con 0x559dcb6849d0 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 20 client.15102 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:27.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb6a4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 4 client.15102 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.139+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dcb5e5430 con 0x559dcb6a33a0 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.140+0000 7f793cbf9700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb6a4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.140+0000 7f7917fff700 10 client.15102 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.140+0000 7f793cbf9700 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb6a4810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e9771a656f17bd9 server_cookie=bcbbd116aa4c046f in_seq=0 out_seq=0 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f7917fff700 1 -- 172.21.15.96:0/3672190732 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dcb5e5430 con 0x559dcb6a33a0 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f7917fff700 10 client.15102 handle_command_reply: tid=0 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 client.15102 shutdown 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 msgr2=0x559dcb68c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb68c2f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f792800f3e0 tx=0x7f792800f6f0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 shutdown_connections 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb68c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb6a4810 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.147 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb68eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f7917fff700 0 client.15102 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T20:57:27.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 --2- 172.21.15.96:0/3672190732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb68c2f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 >> 172.21.15.96:0/3672190732 conn(0x559dcb5b1840 msgr2=0x559dcb6801b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 shutdown_connections 2024-04-25T20:57:27.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 1 -- 172.21.15.96:0/3672190732 wait complete. 2024-04-25T20:57:27.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.141+0000 7f79496e6b80 20 client.15102 trim_cache size 0 max 16384 2024-04-25T20:57:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.161+0000 7f79496e6b80 1 Processor -- start 2024-04-25T20:57:27.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.161+0000 7f79496e6b80 1 -- start start 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.161+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb69f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.161+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb6adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.161+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb6ae1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f79496e6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6a2b50 con 0x559dcb6849d0 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f79496e6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6a3220 con 0x559dcb6840b0 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f79496e6b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6a38f0 con 0x559dcb68f880 2024-04-25T20:57:27.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb6adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb6adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59736/0 (socket says 172.21.15.96:59736) 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 -- 172.21.15.96:0/4134416394 learned_addr learned my addr 172.21.15.96:0/4134416394 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 -- 172.21.15.96:0/4134416394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 msgr2=0x559dcb6ae1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f793cbf9700 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb69f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb6ae1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 -- 172.21.15.96:0/4134416394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 msgr2=0x559dcb69f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb69f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 -- 172.21.15.96:0/4134416394 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcb685b40 con 0x559dcb6849d0 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f793cbf9700 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb69f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f792ffff700 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb6adc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f7928005b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b94960f61bd9c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f791e7fc700 1 -- 172.21.15.96:0/4134416394 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928033070 con 0x559dcb6849d0 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f791e7fc700 1 -- 172.21.15.96:0/4134416394 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792802a040 con 0x559dcb6849d0 2024-04-25T20:57:27.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.162+0000 7f791e7fc700 1 -- 172.21.15.96:0/4134416394 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7928004a20 con 0x559dcb6849d0 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 -- 172.21.15.96:0/4134416394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 msgr2=0x559dcb6adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb6adc70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7928000c00 tx=0x7f7928005b90 comp rx=0 tx=0).stop 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 -- 172.21.15.96:0/4134416394 shutdown_connections 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb68f880 0x559dcb6ae1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb69f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 --2- 172.21.15.96:0/4134416394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb6849d0 0x559dcb6adc70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 -- 172.21.15.96:0/4134416394 >> 172.21.15.96:0/4134416394 conn(0x559dcb5b1840 msgr2=0x559dcb68ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 -- 172.21.15.96:0/4134416394 shutdown_connections 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.163+0000 7f79496e6b80 1 -- 172.21.15.96:0/4134416394 wait complete. 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 -- start start 2024-04-25T20:57:27.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 Processor -- start 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb689030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6849d0 0x559dcb689570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb689ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6a4690 con 0x559dcb68f880 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6a3fc0 con 0x559dcb6840b0 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x559dcb6a4d60 con 0x559dcb6849d0 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f79496e6b80 10 client.0 _ll_register_callbacks cb 0x559dcb5708d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f793d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb689ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f793d3fa700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb689ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59740/0 (socket says 172.21.15.96:59740) 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f793d3fa700 1 -- 172.21.15.96:0/2550266078 learned_addr learned my addr 172.21.15.96:0/2550266078 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f792ffff700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6849d0 0x559dcb689570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.164+0000 7f793cbf9700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb689030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793d3fa700 1 -- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6849d0 msgr2=0x559dcb689570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793d3fa700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6849d0 0x559dcb689570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793d3fa700 1 -- 172.21.15.96:0/2550266078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 msgr2=0x559dcb689030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:27.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793d3fa700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb689030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793d3fa700 1 -- 172.21.15.96:0/2550266078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcb6abf50 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793cbf9700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb689030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f793d3fa700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb689ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79300a6030 tx=0x7f7930007eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5786c5d94434ddc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f7917fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f7917fff700 1 -- 172.21.15.96:0/2550266078 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793002a040 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f792802a040 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f7917fff700 1 -- 172.21.15.96:0/2550266078 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f793009dbe0 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f7917fff700 1 -- 172.21.15.96:0/2550266078 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f79300a6560 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f79496e6b80 10 client.15108 fetch_fsmap learned FSMap version 25 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559dcb70a7c0 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.165+0000 7f7917fff700 1 -- 172.21.15.96:0/2550266078 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792802a040 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f7917fff700 1 -- 172.21.15.96:0/2550266078 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7930011070 con 0x559dcb68f880 2024-04-25T20:57:27.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f79496e6b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f79496e6b80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f79496e6b80 20 client.15108 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb70df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f79496e6b80 4 client.15108 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "status"}) -- 0x559dcb5df4d0 con 0x559dcb6a33a0 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f793cbf9700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb70df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f7917fff700 10 client.15108 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.166+0000 7f793cbf9700 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb70df20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6718b903f3d43927 server_cookie=549b1416d01cf108 in_seq=0 out_seq=0 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f7917fff700 1 -- 172.21.15.96:0/2550266078 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x559dcb5df4d0 con 0x559dcb6a33a0 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f7917fff700 10 client.15108 handle_command_reply: tid=0 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 client.15108 shutdown 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 msgr2=0x559dcb689ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb689ab0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f79300a6030 tx=0x7f7930007eb0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 shutdown_connections 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559dcb68f880 0x559dcb689ab0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x559dcb6a33a0 0x559dcb70df20 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559dcb6840b0 0x559dcb689030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 --2- 172.21.15.96:0/2550266078 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x559dcb6849d0 0x559dcb689570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f7917fff700 0 client.15108 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 >> 172.21.15.96:0/2550266078 conn(0x559dcb5b1840 msgr2=0x559dcb6872c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 shutdown_connections 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 1 -- 172.21.15.96:0/2550266078 wait complete. 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.167+0000 7f79496e6b80 20 client.15108 trim_cache size 0 max 16384 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:57:27.172 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24442, 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 163.85145650199999, 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 93, 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 203.74331549999999 2024-04-25T20:57:27.173 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f79200a3c40 msgr2=0x7f79200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f79200a3c40 0x7f79200a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f79340097b0 tx=0x7f7934006d20 comp rx=0 tx=0).stop 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 msgr2=0x7f793819d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 0x7f793819d9f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f79300062a0 tx=0x7f7930004ed0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 shutdown_connections 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f79200a3c40 0x7f79200a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7938108a50 0x7f793819d9f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7938102a90 0x7f7938199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 --2- 172.21.15.96:0/1406238578 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7938103450 0x7f79381996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:27.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 >> 172.21.15.96:0/1406238578 conn(0x7f7938079ba0 msgr2=0x7f79381017e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:27.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 shutdown_connections 2024-04-25T20:57:27.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:27.168+0000 7f793ee5d700 1 -- 172.21.15.96:0/1406238578 wait complete. 2024-04-25T20:57:27.928 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T20:57:28.163 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:28.630 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:28 smithi096 ceph-mon[29288]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:28 smithi097 ceph-mon[32667]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:28.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:28 smithi139 ceph-mon[33027]: pgmap v392: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 -- 172.21.15.96:0/1629509871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760107990 msgr2=0x7f9760107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 --2- 172.21.15.96:0/1629509871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760107990 0x7f9760107d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f975800b0a0 tx=0x7f975800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 -- 172.21.15.96:0/1629509871 shutdown_connections 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 --2- 172.21.15.96:0/1629509871 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 0x7f976010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 --2- 172.21.15.96:0/1629509871 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760107990 0x7f9760107d70 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 --2- 172.21.15.96:0/1629509871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9760101940 0x7f9760101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.854 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 -- 172.21.15.96:0/1629509871 >> 172.21.15.96:0/1629509871 conn(0x7f97600788f0 msgr2=0x7f9760078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 -- 172.21.15.96:0/1629509871 shutdown_connections 2024-04-25T20:57:29.855 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 -- 172.21.15.96:0/1629509871 wait complete. 2024-04-25T20:57:29.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.853+0000 7f97681d9700 1 Processor -- start 2024-04-25T20:57:29.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 -- start start 2024-04-25T20:57:29.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 0x7f9760199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 0x7f9760199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.856 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9760107990 0x7f976019db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9760111b60 con 0x7f9760101940 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f97601119e0 con 0x7f9760107990 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9760111ce0 con 0x7f9760102300 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 0x7f9760199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 0x7f9760199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59754/0 (socket says 172.21.15.96:59754) 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 -- 172.21.15.96:0/1782682154 learned_addr learned my addr 172.21.15.96:0/1782682154 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9766776700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9760107990 0x7f976019db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765774700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 0x7f9760199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 -- 172.21.15.96:0/1782682154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 msgr2=0x7f9760199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 0x7f9760199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 -- 172.21.15.96:0/1782682154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9760107990 msgr2=0x7f976019db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9760107990 0x7f976019db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 -- 172.21.15.96:0/1782682154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976019e240 con 0x7f9760101940 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765774700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 0x7f9760199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9765f75700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 0x7f9760199230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9758006070 tx=0x7f9758004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9756ffd700 1 -- 172.21.15.96:0/1782682154 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758021070 con 0x7f9760101940 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f9756ffd700 1 -- 172.21.15.96:0/1782682154 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9758007df0 con 0x7f9760101940 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.854+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976019e4d0 con 0x7f9760101940 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.855+0000 7f9756ffd700 1 -- 172.21.15.96:0/1782682154 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975801d450 con 0x7f9760101940 2024-04-25T20:57:29.858 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.855+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97601a5db0 con 0x7f9760101940 2024-04-25T20:57:29.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.857+0000 7f9756ffd700 1 -- 172.21.15.96:0/1782682154 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f97580040b0 con 0x7f9760101940 2024-04-25T20:57:29.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.857+0000 7f9756ffd700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f974c0a3cc0 0x7f974c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.857+0000 7f9765774700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f974c0a3cc0 0x7f974c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.860 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.857+0000 7f9756ffd700 1 -- 172.21.15.96:0/1782682154 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f975800f020 con 0x7f9760101940 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9765774700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f974c0a3cc0 0x7f974c0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 Processor -- start 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 -- start start 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae8be0 0x55e70cae8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caed450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caeddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e70ca36ce0 con 0x55e70caed990 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e70c9dd910 con 0x55e70cae9500 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.858+0000 7f9772a62b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e70ca32ca0 con 0x55e70cae8be0 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caeddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caeddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59760/0 (socket says 172.21.15.96:59760) 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 -- 172.21.15.96:0/633903289 learned_addr learned my addr 172.21.15.96:0/633903289 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9765f75700 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae8be0 0x55e70cae8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 -- 172.21.15.96:0/633903289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae8be0 msgr2=0x55e70cae8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae8be0 0x55e70cae8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 -- 172.21.15.96:0/633903289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 msgr2=0x55e70caed450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caed450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 -- 172.21.15.96:0/633903289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70c984880 con 0x55e70caed990 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9765f75700 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae8be0 0x55e70cae8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f9766776700 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caeddf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b810 tx=0x7f975c00bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f27424b18462def server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f974bfff700 1 -- 172.21.15.96:0/633903289 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c014620 con 0x55e70caed990 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f974bfff700 1 -- 172.21.15.96:0/633903289 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c014c70 con 0x55e70caed990 2024-04-25T20:57:29.862 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.859+0000 7f974bfff700 1 -- 172.21.15.96:0/633903289 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c012d10 con 0x55e70caed990 2024-04-25T20:57:29.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 -- 172.21.15.96:0/633903289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 msgr2=0x55e70caeddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caeddf0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b810 tx=0x7f975c00bb20 comp rx=0 tx=0).stop 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 -- 172.21.15.96:0/633903289 shutdown_connections 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caeddf0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae8be0 0x55e70cae8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 --2- 172.21.15.96:0/633903289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caed450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 -- 172.21.15.96:0/633903289 >> 172.21.15.96:0/633903289 conn(0x55e70ca11640 msgr2=0x55e70cadf820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 -- 172.21.15.96:0/633903289 shutdown_connections 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 -- 172.21.15.96:0/633903289 wait complete. 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 -- start start 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.860+0000 7f9772a62b80 1 Processor -- start 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 0x55e70caf4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caf2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70caf2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e70caf5a20 con 0x55e70cae8be0 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e70c9d5b80 con 0x55e70cae9500 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e70caf5ba0 con 0x55e70caed990 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9772a62b80 10 client.0 _ll_register_callbacks cb 0x55e70c9cf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 0x55e70caf4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 0x55e70caf4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59764/0 (socket says 172.21.15.96:59764) 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 -- 172.21.15.96:0/3294778260 learned_addr learned my addr 172.21.15.96:0/3294778260 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765774700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caf2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9766776700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70caf2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 -- 172.21.15.96:0/3294778260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 msgr2=0x55e70caf2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70caf2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 -- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 msgr2=0x55e70caf2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caf2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 -- 172.21.15.96:0/3294778260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70c9ee220 con 0x55e70cae8be0 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9766776700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70caf2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:29.867 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f9765f75700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 0x55e70caf4e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f97580a64a0 tx=0x7f9758004450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2c7a9dfb9bb2de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.861+0000 7f97497fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f97497fa700 1 -- 172.21.15.96:0/3294778260 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97580a21a0 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c014c70 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f97497fa700 1 -- 172.21.15.96:0/3294778260 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f97580074e0 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f97497fa700 1 -- 172.21.15.96:0/3294778260 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97580a69d0 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 10 client.15120 fetch_fsmap learned FSMap version 25 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e70cb00230 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f97497fa700 1 -- 172.21.15.96:0/3294778260 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c014c70 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f97497fa700 1 -- 172.21.15.96:0/3294778260 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9758011070 con 0x55e70cae8be0 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 20 client.15120 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb02140 0x55e70cb03c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 4 client.15120 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:57:29.868 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e70ca444d0 con 0x55e70cb02140 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.862+0000 7f9765774700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb02140 0x55e70cb03c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.863+0000 7f97497fa700 10 client.15120 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.863+0000 7f9765774700 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb02140 0x55e70cb03c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383a1205f803881 server_cookie=745079684d655143 in_seq=0 out_seq=0 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f97497fa700 1 -- 172.21.15.96:0/3294778260 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e70ca444d0 con 0x55e70cb02140 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f97497fa700 10 client.15120 handle_command_reply: tid=0 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 client.15120 shutdown 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 msgr2=0x55e70caf4e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 0x55e70caf4e30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f97580a64a0 tx=0x7f9758004450 comp rx=0 tx=0).stop 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 shutdown_connections 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb02140 0x55e70cb03c10 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70caf2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae8be0 0x55e70caf4e30 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f97497fa700 0 client.15120 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 --2- 172.21.15.96:0/3294778260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae9500 0x55e70caf2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.869 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 >> 172.21.15.96:0/3294778260 conn(0x55e70ca11640 msgr2=0x55e70caeedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 shutdown_connections 2024-04-25T20:57:29.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 1 -- 172.21.15.96:0/3294778260 wait complete. 2024-04-25T20:57:29.870 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.864+0000 7f9772a62b80 20 client.15120 trim_cache size 0 max 16384 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 Processor -- start 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 -- start start 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cae3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae9500 0x55e70caf4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e70cb01fc0 con 0x55e70caed990 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e70cb02620 con 0x55e70cae8be0 2024-04-25T20:57:29.886 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9772a62b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e70cb02cf0 con 0x55e70cae9500 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9766776700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9766776700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caef4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59770/0 (socket says 172.21.15.96:59770) 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9766776700 1 -- 172.21.15.96:0/2426691850 learned_addr learned my addr 172.21.15.96:0/2426691850 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.884+0000 7f9765f75700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cae3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9765774700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae9500 0x55e70caf4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9766776700 1 -- 172.21.15.96:0/2426691850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae9500 msgr2=0x55e70caf4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9766776700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae9500 0x55e70caf4970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9766776700 1 -- 172.21.15.96:0/2426691850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 msgr2=0x55e70cae3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9766776700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cae3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9766776700 1 -- 172.21.15.96:0/2426691850 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e70caf06c0 con 0x55e70caed990 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9765f75700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cae3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9765774700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae9500 0x55e70caf4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f9766776700 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caef4e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b760 tx=0x7f975c00b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bac4f422c77f3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f974b7fe700 1 -- 172.21.15.96:0/2426691850 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c0123e0 con 0x55e70caed990 2024-04-25T20:57:29.887 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f974b7fe700 1 -- 172.21.15.96:0/2426691850 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c012e10 con 0x55e70caed990 2024-04-25T20:57:29.888 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.885+0000 7f974b7fe700 1 -- 172.21.15.96:0/2426691850 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975c01e440 con 0x55e70caed990 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 -- 172.21.15.96:0/2426691850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 msgr2=0x55e70caef4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caef4e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f975c00b760 tx=0x7f975c00b240 comp rx=0 tx=0).stop 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 -- 172.21.15.96:0/2426691850 shutdown_connections 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70caed990 0x55e70caef4e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cae3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2426691850 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70cae9500 0x55e70caf4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 -- 172.21.15.96:0/2426691850 >> 172.21.15.96:0/2426691850 conn(0x55e70ca11640 msgr2=0x55e70cae35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 -- 172.21.15.96:0/2426691850 shutdown_connections 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 -- 172.21.15.96:0/2426691850 wait complete. 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 -- start start 2024-04-25T20:57:29.891 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.886+0000 7f9772a62b80 1 Processor -- start 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cb0d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 0x55e70cb0d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70cb06030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e70cb03a90 con 0x55e70cae9500 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55e70cb033c0 con 0x55e70cae8be0 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55e70cb04160 con 0x55e70caed990 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 10 client.0 _ll_register_callbacks cb 0x55e70c9cf900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 0x55e70cb0d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 0x55e70cb0d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59772/0 (socket says 172.21.15.96:59772) 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 -- 172.21.15.96:0/2655536536 learned_addr learned my addr 172.21.15.96:0/2655536536 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9766776700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70cb06030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765f75700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cb0d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 -- 172.21.15.96:0/2655536536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 msgr2=0x55e70cb06030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.892 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70cb06030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 -- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 msgr2=0x55e70cb0d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cb0d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 -- 172.21.15.96:0/2655536536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e70cb06730 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765f75700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cb0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9765774700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 0x55e70cb0d5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fa0 tx=0x7f9750004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9389cb182b72dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f97497fa700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f97497fa700 1 -- 172.21.15.96:0/2655536536 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975002a070 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f97497fa700 1 -- 172.21.15.96:0/2655536536 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f9750026800 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.887+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f975c012e10 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f97497fa700 1 -- 172.21.15.96:0/2655536536 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9750036050 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 10 client.15126 fetch_fsmap learned FSMap version 25 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e70cb6a2d0 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f97497fa700 1 -- 172.21.15.96:0/2655536536 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975c012e10 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f97497fa700 1 -- 172.21.15.96:0/2655536536 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f975000f7f0 con 0x55e70cae9500 2024-04-25T20:57:29.893 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 25 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 20 client.15126 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb6eed0 0x55e70cb6dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 4 client.15126 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "status"}) -- 0x55e70ca3e570 con 0x55e70cb6eed0 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.888+0000 7f9765f75700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb6eed0 0x55e70cb6dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f97497fa700 10 client.15126 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9765f75700 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb6eed0 0x55e70cb6dbe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=259ff05a383a31e8 server_cookie=ff1c135f1c0b4fec in_seq=0 out_seq=0 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f97497fa700 1 -- 172.21.15.96:0/2655536536 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e70ca3e570 con 0x55e70cb6eed0 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f97497fa700 10 client.15126 handle_command_reply: tid=0 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 client.15126 shutdown 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 msgr2=0x55e70cb0d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.894 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 0x55e70cb0d5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fa0 tx=0x7f9750004750 comp rx=0 tx=0).stop 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 shutdown_connections 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55e70cb6eed0 0x55e70cb6dbe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55e70caed990 0x55e70cb06030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e70cae8be0 0x55e70cb0d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 --2- 172.21.15.96:0/2655536536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e70cae9500 0x55e70cb0d5d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 >> 172.21.15.96:0/2655536536 conn(0x55e70ca11640 msgr2=0x55e70caec720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f97497fa700 0 client.15126 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 shutdown_connections 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.889+0000 7f9772a62b80 1 -- 172.21.15.96:0/2655536536 wait complete. 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f9772a62b80 20 client.15126 trim_cache size 0 max 16384 2024-04-25T20:57:29.895 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24443, 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 178.49560143799999, 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 25, 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 93, 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 91, 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 196.41520033899999 2024-04-25T20:57:29.896 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f974c0a3cc0 msgr2=0x7f974c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f974c0a3cc0 0x7f974c0a6180 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9750005fd0 tx=0x7f9750005dc0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 msgr2=0x7f9760199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 0x7f9760199230 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f9758006070 tx=0x7f9758004ef0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 shutdown_connections 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f974c0a3cc0 0x7f974c0a6180 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9760107990 0x7f976019db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.897 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9760101940 0x7f9760199230 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 --2- 172.21.15.96:0/1782682154 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9760102300 0x7f9760199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:29.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 >> 172.21.15.96:0/1782682154 conn(0x7f97600788f0 msgr2=0x7f976006a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:29.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 shutdown_connections 2024-04-25T20:57:29.898 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:29.890+0000 7f97681d9700 1 -- 172.21.15.96:0/1782682154 wait complete. 2024-04-25T20:57:30.532 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:30 smithi096 ceph-mon[29288]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:30.559 INFO:teuthology.run_tasks:Running task sequential... 2024-04-25T20:57:30.569 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-25T20:57:30.569 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-25T20:57:30.570 INFO:teuthology.task.print:Enabling mgr modules 2024-04-25T20:57:30.570 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-25T20:57:30.570 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T20:57:30.570 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:57:30.571 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.670+0000 7f3840795640 1 -- 172.21.15.96:0/359095009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 msgr2=0x7f3838102400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.670+0000 7f3840795640 1 --2- 172.21.15.96:0/359095009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838102400 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3834009ca0 tx=0x7f383402a8c0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- 172.21.15.96:0/359095009 shutdown_connections 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 --2- 172.21.15.96:0/359095009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3838103b90 0x7f383810a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 --2- 172.21.15.96:0/359095009 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f38381031d0 0x7f3838103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 --2- 172.21.15.96:0/359095009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838102400 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- 172.21.15.96:0/359095009 >> 172.21.15.96:0/359095009 conn(0x7f38380fd570 msgr2=0x7f38380ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- 172.21.15.96:0/359095009 shutdown_connections 2024-04-25T20:57:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- 172.21.15.96:0/359095009 wait complete. 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 Processor -- start 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- start start 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38381031d0 0x7f3838198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3838103b90 0x7f383819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f383810db60 con 0x7f3838101fe0 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f383810d9e0 con 0x7f38381031d0 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f3840795640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f383810dce0 con 0x7f3838103b90 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f383e531640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f383e531640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59796/0 (socket says 172.21.15.96:59796) 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.671+0000 7f383e531640 1 -- 172.21.15.96:0/421007185 learned_addr learned my addr 172.21.15.96:0/421007185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383ed32640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3838103b90 0x7f383819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:30.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383e531640 1 -- 172.21.15.96:0/421007185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3838103b90 msgr2=0x7f383819f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383e531640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3838103b90 0x7f383819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383e531640 1 -- 172.21.15.96:0/421007185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38381031d0 msgr2=0x7f3838198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383dd30640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38381031d0 0x7f3838198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383e531640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38381031d0 0x7f3838198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383e531640 1 -- 172.21.15.96:0/421007185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f383819fe20 con 0x7f3838101fe0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f383e531640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838198200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3834003c20 tx=0x7f3834003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3834004240 con 0x7f3838101fe0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38381a00b0 con 0x7f3838101fe0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38340317c0 con 0x7f3838101fe0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38340305c0 con 0x7f3838101fe0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.672+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38381a0680 con 0x7f3838101fe0 2024-04-25T20:57:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.673+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3838109300 con 0x7f3838101fe0 2024-04-25T20:57:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.674+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f383402f6f0 con 0x7f3838101fe0 2024-04-25T20:57:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.674+0000 7f382f7fe640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f38240a3c50 0x7f38240a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.675+0000 7f383dd30640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f38240a3c50 0x7f38240a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.675+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f38340f1c30 con 0x7f3838101fe0 2024-04-25T20:57:30.678 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.675+0000 7f383dd30640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f38240a3c50 0x7f38240a6110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3838199710 tx=0x7f3828008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:30.680 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.679+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38340be6b0 con 0x7f3838101fe0 2024-04-25T20:57:30.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:30 smithi097 ceph-mon[32667]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:30 smithi139 ceph-mon[33027]: pgmap v393: 97 pgs: 97 active+clean; 640 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-04-25T20:57:30.905 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:30.904+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f3838062b50 con 0x7f3838101fe0 2024-04-25T20:57:31.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.383+0000 7f382f7fe640 1 -- 172.21.15.96:0/421007185 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f38340098b0 con 0x7f3838101fe0 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f38240a3c50 msgr2=0x7f38240a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f38240a3c50 0x7f38240a6110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3838199710 tx=0x7f3828008040 comp rx=0 tx=0).stop 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 msgr2=0x7f3838198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838198200 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3834003c20 tx=0x7f3834003c50 comp rx=0 tx=0).stop 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 shutdown_connections 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3838103b90 0x7f383819f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f38381031d0 0x7f3838198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f38240a3c50 0x7f38240a6110 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 --2- 172.21.15.96:0/421007185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3838101fe0 0x7f3838198200 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.385+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 >> 172.21.15.96:0/421007185 conn(0x7f38380fd570 msgr2=0x7f38381064d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.386+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 shutdown_connections 2024-04-25T20:57:31.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.386+0000 7f3840795640 1 -- 172.21.15.96:0/421007185 wait complete. 2024-04-25T20:57:31.404 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-25T20:57:31.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 -- 172.21.15.96:0/1473236357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102040 msgr2=0x7fb0181024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1473236357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102040 0x7fb0181024c0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb00c009900 tx=0x7fb00c02a410 comp rx=0 tx=0).stop 2024-04-25T20:57:31.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 -- 172.21.15.96:0/1473236357 shutdown_connections 2024-04-25T20:57:31.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1473236357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018102a00 0x7fb018109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.514 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1473236357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102040 0x7fb0181024c0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1473236357 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb018100e60 0x7fb018101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 -- 172.21.15.96:0/1473236357 >> 172.21.15.96:0/1473236357 conn(0x7fb0180fc420 msgr2=0x7fb0180fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:31.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 -- 172.21.15.96:0/1473236357 shutdown_connections 2024-04-25T20:57:31.515 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.513+0000 7fb01df5d640 1 -- 172.21.15.96:0/1473236357 wait complete. 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 Processor -- start 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 -- start start 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018100e60 0x7fb018198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb018102040 0x7fb0181986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 0x7fb01819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb01810db30 con 0x7fb018102a00 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb01810d9b0 con 0x7fb018100e60 2024-04-25T20:57:31.517 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb01df5d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb01810dcb0 con 0x7fb018102040 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb017fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 0x7fb01819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb017fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 0x7fb01819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59834/0 (socket says 172.21.15.96:59834) 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.514+0000 7fb017fff640 1 -- 172.21.15.96:0/1335622053 learned_addr learned my addr 172.21.15.96:0/1335622053 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018100e60 0x7fb018198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb016ffd640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb018102040 0x7fb0181986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 -- 172.21.15.96:0/1335622053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb018102040 msgr2=0x7fb0181986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb018102040 0x7fb0181986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 -- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 msgr2=0x7fb01819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 0x7fb01819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 -- 172.21.15.96:0/1335622053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01819fd80 con 0x7fb018100e60 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb017fff640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 0x7fb01819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb0177fe640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018100e60 0x7fb018198160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb018101bb0 tx=0x7fb00800dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:31.518 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb008017070 con 0x7fb018100e60 2024-04-25T20:57:31.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0181a0070 con 0x7fb018100e60 2024-04-25T20:57:31.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb008009c30 con 0x7fb018100e60 2024-04-25T20:57:31.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb00800f620 con 0x7fb018100e60 2024-04-25T20:57:31.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.515+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0181a05e0 con 0x7fb018100e60 2024-04-25T20:57:31.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.516+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb018108280 con 0x7fb018100e60 2024-04-25T20:57:31.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.518+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb00801a070 con 0x7fb018100e60 2024-04-25T20:57:31.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.519+0000 7fb014ff9640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0000a3ca0 0x7fb0000a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.519+0000 7fb016ffd640 1 -- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0000a3ca0 msgr2=0x7fb0000a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:31.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.519+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fb0080cff50 con 0x7fb018100e60 2024-04-25T20:57:31.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.519+0000 7fb016ffd640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0000a3ca0 0x7fb0000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:31.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.522+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb00809c380 con 0x7fb018100e60 2024-04-25T20:57:31.690 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:31 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/421007185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T20:57:31.690 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: ignoring --setuser ceph since I am not root 2024-04-25T20:57:31.691 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: ignoring --setgroup ceph since I am not root 2024-04-25T20:57:31.691 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:31.577+0000 7f85d3b39200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:57:31.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.708+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb018062b50 con 0x7fb018100e60 2024-04-25T20:57:31.714 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.713+0000 7fb014ff9640 1 -- 172.21.15.96:0/1335622053 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fb0080a0680 con 0x7fb018100e60 2024-04-25T20:57:31.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0000a3ca0 msgr2=0x7fb0000a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:31.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0000a3ca0 0x7fb0000a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018100e60 msgr2=0x7fb018198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018100e60 0x7fb018198160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb018101bb0 tx=0x7fb00800dde0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 shutdown_connections 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fb0000a3ca0 0x7fb0000a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb018102a00 0x7fb01819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb018102040 0x7fb0181986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 --2- 172.21.15.96:0/1335622053 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb018100e60 0x7fb018198160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 >> 172.21.15.96:0/1335622053 conn(0x7fb0180fc420 msgr2=0x7fb018105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 shutdown_connections 2024-04-25T20:57:31.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.716+0000 7fb01df5d640 1 -- 172.21.15.96:0/1335622053 wait complete. 2024-04-25T20:57:31.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-25T20:57:31.748 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/421007185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T20:57:31.748 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: ignoring --setuser ceph since I am not root 2024-04-25T20:57:31.748 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: ignoring --setgroup ceph since I am not root 2024-04-25T20:57:31.748 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-mgr[34077]: -- 172.21.15.97:0/2605627910 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561cbb4845a0 con 0x561cbb459000 2024-04-25T20:57:31.748 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:31.565+0000 7effd67c0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:57:31.748 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:31.747+0000 7effd67c0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:31.749 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/421007185' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-25T20:57:31.749 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: ignoring --setuser ceph since I am not root 2024-04-25T20:57:31.749 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: ignoring --setgroup ceph since I am not root 2024-04-25T20:57:31.749 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-mgr[34889]: -- 172.21.15.139:0/3281493322 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4a851a5a0 con 0x55c4a84ee800 2024-04-25T20:57:31.749 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:31.565+0000 7f6142174200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T20:57:31.749 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:31.748+0000 7f6142174200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.832+0000 7fa83339b640 1 -- 172.21.15.96:0/1220746628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 msgr2=0x7fa82c107750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.832+0000 7fa83339b640 1 --2- 172.21.15.96:0/1220746628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c107750 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa828009900 tx=0x7fa82802a410 comp rx=0 tx=0).stop 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- 172.21.15.96:0/1220746628 shutdown_connections 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 --2- 172.21.15.96:0/1220746628 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c107c90 0x7fa82c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 --2- 172.21.15.96:0/1220746628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c107750 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 --2- 172.21.15.96:0/1220746628 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa82c102a30 0x7fa82c104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- 172.21.15.96:0/1220746628 >> 172.21.15.96:0/1220746628 conn(0x7fa82c0fc400 msgr2=0x7fa82c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- 172.21.15.96:0/1220746628 shutdown_connections 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- 172.21.15.96:0/1220746628 wait complete. 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 Processor -- start 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- start start 2024-04-25T20:57:31.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c102a30 0x7fa82c1023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c1009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa82c107c90 0x7fa82c100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c10dab0 con 0x7fa82c105360 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c10d930 con 0x7fa82c102a30 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa83339b640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa82c10dc30 con 0x7fa82c107c90 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.833+0000 7fa830936640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c1009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c1009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59842/0 (socket says 172.21.15.96:59842) 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 -- 172.21.15.96:0/1363240749 learned_addr learned my addr 172.21.15.96:0/1363240749 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa831938640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa82c107c90 0x7fa82c100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa831137640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c102a30 0x7fa82c1023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 -- 172.21.15.96:0/1363240749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa82c107c90 msgr2=0x7fa82c100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa82c107c90 0x7fa82c100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 -- 172.21.15.96:0/1363240749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c102a30 msgr2=0x7fa82c1023a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:31.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c102a30 0x7fa82c1023a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 -- 172.21.15.96:0/1363240749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82c1017b0 con 0x7fa82c105360 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa830936640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c1009e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa828006010 tx=0x7fa8280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82803f030 con 0x7fa82c105360 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82c1a2210 con 0x7fa82c105360 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8280318f0 con 0x7fa82c105360 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828035070 con 0x7fa82c105360 2024-04-25T20:57:31.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.834+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa82c1a27b0 con 0x7fa82c105360 2024-04-25T20:57:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.836+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa82c0fdef0 con 0x7fa82c105360 2024-04-25T20:57:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.837+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa828042030 con 0x7fa82c105360 2024-04-25T20:57:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.838+0000 7fa8227fc640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8180a3ca0 0x7fa8180a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.838+0000 7fa831137640 1 -- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8180a3ca0 msgr2=0x7fa8180a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.838+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7fa8280f0870 con 0x7fa82c105360 2024-04-25T20:57:31.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.838+0000 7fa831137640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8180a3ca0 0x7fa8180a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:31.843 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:31.842+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa8280bcca0 con 0x7fa82c105360 2024-04-25T20:57:31.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:31.773+0000 7f85d3b39200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:31.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:31.888+0000 7f85d3b39200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:57:32.029 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.029+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fa82c062b50 con 0x7fa82c105360 2024-04-25T20:57:32.035 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.035+0000 7fa8227fc640 1 -- 172.21.15.96:0/1363240749 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7fa8280c0fa0 con 0x7fa82c105360 2024-04-25T20:57:32.038 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8180a3ca0 msgr2=0x7fa8180a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8180a3ca0 0x7fa8180a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 msgr2=0x7fa82c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c1009e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa828006010 tx=0x7fa8280042f0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 shutdown_connections 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa82c107c90 0x7fa82c100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7fa8180a3ca0 0x7fa8180a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa82c105360 0x7fa82c1009e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 --2- 172.21.15.96:0/1363240749 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa82c102a30 0x7fa82c1023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 >> 172.21.15.96:0/1363240749 conn(0x7fa82c0fc400 msgr2=0x7fa82c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 shutdown_connections 2024-04-25T20:57:32.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:32.037+0000 7fa83339b640 1 -- 172.21.15.96:0/1363240749 wait complete. 2024-04-25T20:57:32.050 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T20:57:32.073 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T20:57:32.073 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-25T20:57:32.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:32.073 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T20:57:32.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:32.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T20:57:32.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:31.850+0000 7effd67c0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:57:32.122 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:31.851+0000 7f6142174200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T20:57:32.156 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-04-25T20:57:32.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T20:57:32.157 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'nowsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-04-25T20:57:32.158 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T20:57:32.158 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T20:57:32.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-25T20:57:32.306 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:32 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:32.215+0000 7f85d3b39200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:57:32.306 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:32 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:32.304+0000 7f85d3b39200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:57:32.458 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:32.582 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:32 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/421007185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T20:57:32.582 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:32 smithi096 ceph-mon[29288]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T20:57:32.582 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:32 smithi096 ceph-mon[29288]: from='client.? ' entity='client.admin' 2024-04-25T20:57:32.582 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:32 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1363240749' entity='client.admin' 2024-04-25T20:57:32.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:32 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:32.156+0000 7effd67c0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:57:32.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:32 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:32.239+0000 7effd67c0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:57:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:32 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/421007185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T20:57:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:32 smithi097 ceph-mon[32667]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T20:57:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:32 smithi097 ceph-mon[32667]: from='client.? ' entity='client.admin' 2024-04-25T20:57:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:32 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1363240749' entity='client.admin' 2024-04-25T20:57:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:32 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/421007185' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-25T20:57:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:32 smithi139 ceph-mon[33027]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-25T20:57:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:32 smithi139 ceph-mon[33027]: from='client.? ' entity='client.admin' 2024-04-25T20:57:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:32 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1363240749' entity='client.admin' 2024-04-25T20:57:32.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:32.158+0000 7f6142174200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T20:57:32.623 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:32.242+0000 7f6142174200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T20:57:33.746 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:33.409+0000 7effd67c0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:57:33.746 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:33.577+0000 7effd67c0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:33.761 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:33.420+0000 7f6142174200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:57:33.761 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:33.589+0000 7f6142174200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:33.866 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:33.539+0000 7f85d3b39200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T20:57:33.866 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:33.713+0000 7f85d3b39200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:34.033 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:33.746+0000 7effd67c0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:57:34.051 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:33.760+0000 7f6142174200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:57:34.051 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:34.050+0000 7f6142174200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:57:34.184 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:33.899+0000 7f85d3b39200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T20:57:34.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 -- 172.21.15.96:0/3500239433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 msgr2=0x7f51441021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:34.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 --2- 172.21.15.96:0/3500239433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f51441021a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5130009990 tx=0x7f5130009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 -- 172.21.15.96:0/3500239433 shutdown_connections 2024-04-25T20:57:34.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 --2- 172.21.15.96:0/3500239433 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51441026e0 0x7f514410e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 --2- 172.21.15.96:0/3500239433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5144106990 0x7f5144106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 --2- 172.21.15.96:0/3500239433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f51441021a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 -- 172.21.15.96:0/3500239433 >> 172.21.15.96:0/3500239433 conn(0x7f51440788f0 msgr2=0x7f5144078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:34.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 -- 172.21.15.96:0/3500239433 shutdown_connections 2024-04-25T20:57:34.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.186+0000 7f514b0f4700 1 -- 172.21.15.96:0/3500239433 wait complete. 2024-04-25T20:57:34.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 Processor -- start 2024-04-25T20:57:34.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 -- start start 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f5144199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51441026e0 0x7f514419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5144106990 0x7f51441a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5144111b30 con 0x7f5144101d20 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51441119b0 con 0x7f51441026e0 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f514b0f4700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5144111cb0 con 0x7f5144106990 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f5148e90700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f5144199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f5148e90700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f5144199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59858/0 (socket says 172.21.15.96:59858) 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f5148e90700 1 -- 172.21.15.96:0/1867191146 learned_addr learned my addr 172.21.15.96:0/1867191146 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f5143fff700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51441026e0 0x7f514419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:34.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.187+0000 7f5149691700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5144106990 0x7f51441a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5148e90700 1 -- 172.21.15.96:0/1867191146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5144106990 msgr2=0x7f51441a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5148e90700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5144106990 0x7f51441a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5148e90700 1 -- 172.21.15.96:0/1867191146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51441026e0 msgr2=0x7f514419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5148e90700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51441026e0 0x7f514419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5148e90700 1 -- 172.21.15.96:0/1867191146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51441a5530 con 0x7f5144101d20 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5148e90700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f5144199f70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f513800d950 tx=0x7f513800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5138016070 con 0x7f5144101d20 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51441a5700 con 0x7f5144101d20 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f513800fd40 con 0x7f5144101d20 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513800e3b0 con 0x7f5144101d20 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.188+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51441a5c40 con 0x7f5144101d20 2024-04-25T20:57:34.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.189+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5144103080 con 0x7f5144101d20 2024-04-25T20:57:34.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.190+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f5138019070 con 0x7f5144101d20 2024-04-25T20:57:34.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.191+0000 7f5141ffb700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 0x7f51340a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:34.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.191+0000 7f5143fff700 1 -- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 msgr2=0x7f51340a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:34.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.191+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f51380d0140 con 0x7f5144101d20 2024-04-25T20:57:34.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.191+0000 7f5143fff700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 0x7f51340a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:34.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.195+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f513809cbc0 con 0x7f5144101d20 2024-04-25T20:57:34.323 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:34.033+0000 7effd67c0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:57:34.323 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:34.124+0000 7effd67c0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:57:34.347 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:34.141+0000 7f6142174200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:57:34.348 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:34.346+0000 7f6142174200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:57:34.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.392+0000 7f5143fff700 1 -- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 msgr2=0x7f51340a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:34.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.392+0000 7f5143fff700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 0x7f51340a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:34.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.448+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f514402df80 con 0x7f5144101d20 2024-04-25T20:57:34.449 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:34.206+0000 7f85d3b39200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T20:57:34.450 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:34.302+0000 7f85d3b39200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T20:57:34.450 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.449+0000 7f5141ffb700 1 -- 172.21.15.96:0/1867191146 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+73 (secure 0 0 0) 0x7f513809c4c0 con 0x7f5144101d20 2024-04-25T20:57:34.450 INFO:teuthology.orchestra.run.smithi096.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T20:57:34.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 msgr2=0x7f51340a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:34.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 0x7f51340a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 msgr2=0x7f5144199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:34.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f5144199f70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f513800d950 tx=0x7f513800dc60 comp rx=0 tx=0).stop 2024-04-25T20:57:34.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 shutdown_connections 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f51340a3c90 0x7f51340a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5144106990 0x7f51441a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5144101d20 0x7f5144199f70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 --2- 172.21.15.96:0/1867191146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51441026e0 0x7f514419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 >> 172.21.15.96:0/1867191146 conn(0x7f51440788f0 msgr2=0x7f51440ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 shutdown_connections 2024-04-25T20:57:34.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:34.451+0000 7f514b0f4700 1 -- 172.21.15.96:0/1867191146 wait complete. 2024-04-25T20:57:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:34 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1867191146' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T20:57:34.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:34.322+0000 7effd67c0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:57:34.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:34.516+0000 7effd67c0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:34 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1867191146' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T20:57:34.622 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:34.541+0000 7f6142174200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:34.708 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:34 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1867191146' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-25T20:57:34.709 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:34.514+0000 7f85d3b39200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T20:57:34.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:34.707+0000 7f85d3b39200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi096.front.sepia.ceph.com 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T20:57:35.135 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T20:57:35.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:35.136 DEBUG:teuthology.orchestra.run.smithi096:> ip addr 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 ::1/128 scope host 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bc brd ff:ff:ff:ff:ff:ff 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f0 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:d8:d9:bd brd ff:ff:ff:ff:ff:ff 2024-04-25T20:57:35.152 INFO:teuthology.orchestra.run.smithi096.stdout: altname enp7s0f1 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0c brd ff:ff:ff:ff:ff:ff 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout: link/ether 0c:c4:7a:8f:cc:0d brd ff:ff:ff:ff:ff:ff 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout: inet 172.21.15.96/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft 42062sec preferred_lft 42062sec 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout: inet6 fe80::45d2:c701:9f98:9709/64 scope link noprefixroute 2024-04-25T20:57:35.153 INFO:teuthology.orchestra.run.smithi096.stdout: valid_lft forever preferred_lft forever 2024-04-25T20:57:35.153 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T20:57:35.153 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:57:35.154 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T20:57:35.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add name ceph-brx type bridge 2024-04-25T20:57:35.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr flush dev ceph-brx 2024-04-25T20:57:35.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set ceph-brx up 2024-04-25T20:57:35.154 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T20:57:35.154 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T20:57:35.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:57:35.355 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:35.355 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:35.327+0000 7f85d3b39200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:57:35.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:35.362 DEBUG:teuthology.orchestra.run.smithi096:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T20:57:35.363 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:35.104+0000 7effd67c0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:57:35.386 INFO:teuthology.orchestra.run.smithi096.stdout:1 2024-04-25T20:57:35.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:35.388 DEBUG:teuthology.orchestra.run.smithi096:> route 2024-04-25T20:57:35.452 INFO:teuthology.orchestra.run.smithi096.stdout:Kernel IP routing table 2024-04-25T20:57:35.452 INFO:teuthology.orchestra.run.smithi096.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T20:57:35.452 INFO:teuthology.orchestra.run.smithi096.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T20:57:35.452 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T20:57:35.452 INFO:teuthology.orchestra.run.smithi096.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T20:57:35.452 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:57:35.452 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T20:57:35.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T20:57:35.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T20:57:35.453 DEBUG:teuthology.orchestra.run.smithi096:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T20:57:35.453 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T20:57:35.542 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:57:35.562 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:35.142+0000 7f6142174200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T20:57:35.731 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:35.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:35.738 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T20:57:35.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:35.753 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list-id 2024-04-25T20:57:35.807 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:57:35.807 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T20:57:35.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T20:57:35.807 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T20:57:35.808 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T20:57:35.816 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:35.520+0000 7effd67c0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:57:35.816 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:35.616+0000 7effd67c0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:57:35.860 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:35.561+0000 7f6142174200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:57:35.860 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:35.658+0000 7f6142174200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:57:35.881 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:35.779+0000 7f85d3b39200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T20:57:35.881 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:35.880+0000 7f85d3b39200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T20:57:35.896 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:57:35.918 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:35.924 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T20:57:35.925 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T20:57:36.014 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:57:36.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:36.090 DEBUG:teuthology.orchestra.run.smithi096:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-25T20:57:36.090 DEBUG:teuthology.orchestra.run.smithi096:> set -e 2024-04-25T20:57:36.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set brx.0 up 2024-04-25T20:57:36.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T20:57:36.091 DEBUG:teuthology.orchestra.run.smithi096:> ') 2024-04-25T20:57:36.113 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:35.816+0000 7effd67c0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:57:36.122 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:35.859+0000 7f6142174200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:57:36.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:57:36.146 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:36.102+0000 7f85d3b39200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T20:57:36.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:36.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T20:57:36.166 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T20:57:36.190 INFO:teuthology.orchestra.run.smithi096.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T20:57:36.190 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T20:57:36.191 DEBUG:teuthology.orchestra.run.smithi096:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T20:57:36.246 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-25T20:57:36.246 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-25T20:57:36.246 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/9e3c18f5-af88-4dc8-93e6-d6c5693be9b9 2024-04-25T20:57:36.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:36.246 DEBUG:teuthology.orchestra.run.smithi096:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/9e3c18f5-af88-4dc8-93e6-d6c5693be9b9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T20:57:36.340 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T20:57:36.341 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T20:57:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,nowsync,nofallback 2024-04-25T20:57:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync". 2024-04-25T20:57:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: resolved to: "172.21.15.96:3300,172.21.15.97:3300,172.21.15.139:3300" 2024-04-25T20:57:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: trying mount with new device syntax: 0@b713778a-0344-11ef-bc93-c7b262605968.cephfs=/volumes/qa/sv_1/9e3c18f5-af88-4dc8-93e6-d6c5693be9b9 2024-04-25T20:57:36.447 INFO:teuthology.orchestra.run.smithi096.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,nowsync,name=0,key=0,mon_addr=172.21.15.96:3300/172.21.15.97:3300/172.21.15.139:3300" will pass to kernel 2024-04-25T20:57:36.448 INFO:teuthology.orchestra.run.smithi096.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T20:57:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: You just mounted a file system that supports labels which does not 2024-04-25T20:57:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T20:57:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T20:57:36.449 INFO:teuthology.orchestra.run.smithi096.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T20:57:36.450 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T20:57:36.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T20:57:36.450 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T20:57:36.480 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T20:57:36.569 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T20:57:36.570 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:36.576 INFO:teuthology.orchestra.run.smithi096.stdout:client24731 2024-04-25T20:57:36.576 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:57:36.577 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T20:57:36.592 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T20:57:36.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:36.330+0000 7effd67c0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:57:36.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:36.423+0000 7effd67c0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:57:36.681 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T20:57:36.682 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:36.689 INFO:teuthology.orchestra.run.smithi096.stdout:client24731 2024-04-25T20:57:36.689 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T20:57:36.717 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:36.382+0000 7f6142174200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:57:36.717 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:36.475+0000 7f6142174200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:57:36.737 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-25T20:57:36.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:36.745 INFO:teuthology.orchestra.run.smithi096.stdout:client24731 2024-04-25T20:57:36.745 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b713778a-0344-11ef-bc93-c7b262605968.client24731/status') 2024-04-25T20:57:36.749 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:36.650+0000 7f85d3b39200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T20:57:36.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-25T20:57:36.803 INFO:teuthology.orchestra.run.smithi096.stdout:instance: client.24731 (3)172.21.15.96:0/2326545209 2024-04-25T20:57:36.803 INFO:teuthology.orchestra.run.smithi096.stdout:blocklisted: false 2024-04-25T20:57:36.803 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records in 2024-04-25T20:57:36.803 INFO:teuthology.orchestra.run.smithi096.stderr:0+1 records out 2024-04-25T20:57:36.803 INFO:teuthology.orchestra.run.smithi096.stderr:71 bytes copied, 3.9086e-05 s, 1.8 MB/s 2024-04-25T20:57:36.804 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T20:57:36.810 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T20:57:36.821 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T20:57:36.822 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:57:36.822 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-25T20:57:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 -- 172.21.15.96:0/1241762661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 msgr2=0x7f35941012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 --2- 172.21.15.96:0/1241762661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f35941012a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3584009900 tx=0x7f358402a410 comp rx=0 tx=0).stop 2024-04-25T20:57:36.954 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 -- 172.21.15.96:0/1241762661 shutdown_connections 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 --2- 172.21.15.96:0/1241762661 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3594102a20 0x7f3594109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 --2- 172.21.15.96:0/1241762661 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102060 0x7f35941024e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 --2- 172.21.15.96:0/1241762661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f35941012a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 -- 172.21.15.96:0/1241762661 >> 172.21.15.96:0/1241762661 conn(0x7f35940fc440 msgr2=0x7f35940fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 -- 172.21.15.96:0/1241762661 shutdown_connections 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.951+0000 7f3599c64640 1 -- 172.21.15.96:0/1241762661 wait complete. 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 Processor -- start 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 -- start start 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f3594197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3594102060 0x7f3594197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102a20 0x7f359419e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f359410ca20 con 0x7f3594100e80 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f359410c8a0 con 0x7f3594102a20 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3599c64640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f359410cba0 con 0x7f3594102060 2024-04-25T20:57:36.955 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f3594197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f3594197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52320/0 (socket says 172.21.15.96:52320) 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 -- 172.21.15.96:0/4291990036 learned_addr learned my addr 172.21.15.96:0/4291990036 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3593fff640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102a20 0x7f359419e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f358bfff640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3594102060 0x7f3594197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 -- 172.21.15.96:0/4291990036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3594102060 msgr2=0x7f3594197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3594102060 0x7f3594197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 -- 172.21.15.96:0/4291990036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102a20 msgr2=0x7f359419e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102a20 0x7f359419e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 -- 172.21.15.96:0/4291990036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f359419ec50 con 0x7f3594100e80 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f3593fff640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102a20 0x7f359419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.952+0000 7f35937fe640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f3594197030 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3584006010 tx=0x7f35840042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.953+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f358403f030 con 0x7f3594100e80 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.953+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f359406a720 con 0x7f3594100e80 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.953+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3584031910 con 0x7f3594100e80 2024-04-25T20:57:36.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.953+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3584035070 con 0x7f3594100e80 2024-04-25T20:57:36.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.953+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f359406ac60 con 0x7f3594100e80 2024-04-25T20:57:36.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.954+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f359404fce0 con 0x7f3594100e80 2024-04-25T20:57:36.957 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.956+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3584042030 con 0x7f3594100e80 2024-04-25T20:57:36.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.956+0000 7f35917fa640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 0x7f35800a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:36.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.956+0000 7f358bfff640 1 -- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 msgr2=0x7f35800a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:36.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.956+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f35840f0890 con 0x7f3594100e80 2024-04-25T20:57:36.958 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.956+0000 7f358bfff640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 0x7f35800a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:36.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:36.961+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f35840bccc0 con 0x7f3594100e80 2024-04-25T20:57:37.006 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:36.664+0000 7effd67c0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:57:37.006 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:36.796+0000 7effd67c0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:57:37.006 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:36.890+0000 7effd67c0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:57:37.006 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:37.006+0000 7effd67c0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:37.011 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:36.748+0000 7f85d3b39200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T20:57:37.059 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:36.716+0000 7f6142174200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:57:37.059 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:36.848+0000 7f6142174200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:57:37.059 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:36.943+0000 7f6142174200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:57:37.157 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.157+0000 7f358bfff640 1 -- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 msgr2=0x7f35800a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:37.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.157+0000 7f358bfff640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 0x7f35800a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:37.186 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.185+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f3594107e70 con 0x7f3594100e80 2024-04-25T20:57:37.238 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.237+0000 7f35917fa640 1 -- 172.21.15.96:0/4291990036 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f35840c0fc0 con 0x7f3594100e80 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 msgr2=0x7f35800a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 0x7f35800a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 msgr2=0x7f3594197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f3594197030 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3584006010 tx=0x7f35840042f0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 shutdown_connections 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3594102a20 0x7f359419e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3594102060 0x7f3594197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f35800a3ca0 0x7f35800a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 --2- 172.21.15.96:0/4291990036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3594100e80 0x7f3594197030 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 >> 172.21.15.96:0/4291990036 conn(0x7f35940fc440 msgr2=0x7f35941053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 shutdown_connections 2024-04-25T20:57:37.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.240+0000 7f3599c64640 1 -- 172.21.15.96:0/4291990036 wait complete. 2024-04-25T20:57:37.252 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-25T20:57:37.272 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:37 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4291990036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T20:57:37.272 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:37.010+0000 7f85d3b39200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T20:57:37.272 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:37.149+0000 7f85d3b39200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T20:57:37.272 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:37.251+0000 7f85d3b39200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T20:57:37.305 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:37 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4291990036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 -- 172.21.15.96:0/2034822833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101180 msgr2=0x7f4a741015e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 --2- 172.21.15.96:0/2034822833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101180 0x7f4a741015e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009900 tx=0x7f4a7002a410 comp rx=0 tx=0).stop 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 -- 172.21.15.96:0/2034822833 shutdown_connections 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 --2- 172.21.15.96:0/2034822833 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a74101b20 0x7f4a74101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 --2- 172.21.15.96:0/2034822833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101180 0x7f4a741015e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 --2- 172.21.15.96:0/2034822833 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74100820 0x7f4a74100c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 -- 172.21.15.96:0/2034822833 >> 172.21.15.96:0/2034822833 conn(0x7f4a740fc420 msgr2=0x7f4a740fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 -- 172.21.15.96:0/2034822833 shutdown_connections 2024-04-25T20:57:37.357 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.354+0000 7f4a7c643640 1 -- 172.21.15.96:0/2034822833 wait complete. 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 Processor -- start 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 -- start start 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a74100820 0x7f4a7419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74101180 0x7f4a7419a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101b20 0x7f4a741a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410fd00 con 0x7f4a74101b20 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410fb80 con 0x7f4a74101180 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7c643640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7410fe80 con 0x7f4a74100820 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7abe0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101b20 0x7f4a741a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7a3df640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a74100820 0x7f4a7419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7abe0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101b20 0x7f4a741a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52330/0 (socket says 172.21.15.96:52330) 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a7abe0640 1 -- 172.21.15.96:0/3924665206 learned_addr learned my addr 172.21.15.96:0/3924665206 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:37.358 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.355+0000 7f4a79bde640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74101180 0x7f4a7419a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7a3df640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74101180 msgr2=0x7f4a7419a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7a3df640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4a74101180 0x7f4a7419a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7a3df640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101b20 msgr2=0x7f4a741a18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7a3df640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101b20 0x7f4a741a18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7a3df640 1 -- 172.21.15.96:0/3924665206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a741a1f50 con 0x7f4a74100820 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7abe0640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a74101b20 0x7f4a741a18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7a3df640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a74100820 0x7f4a7419a330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400e9f0 tx=0x7f4a6400eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a6400ccc0 con 0x7f4a74100820 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a741a2240 con 0x7f4a74100820 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a741a27b0 con 0x7f4a74100820 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a640108e0 con 0x7f4a74100820 2024-04-25T20:57:37.359 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.356+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64018750 con 0x7f4a74100820 2024-04-25T20:57:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.357+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a7404fce0 con 0x7f4a74100820 2024-04-25T20:57:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.358+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f4a64010430 con 0x7f4a74100820 2024-04-25T20:57:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.359+0000 7f4a6b7fe640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 0x7f4a600a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.359+0000 7f4a79bde640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 msgr2=0x7f4a600a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:37.360 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.359+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8153+0+0 (secure 0 0 0) 0x7f4a640d0890 con 0x7f4a74100820 2024-04-25T20:57:37.361 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.359+0000 7f4a79bde640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 0x7f4a600a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T20:57:37.368 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:37 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4291990036' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-25T20:57:37.368 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:37.058+0000 7f6142174200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:37.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.369+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4a6409ccc0 con 0x7f4a74100820 2024-04-25T20:57:37.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.560+0000 7f4a79bde640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 msgr2=0x7f4a600a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:37.561 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.560+0000 7f4a79bde640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 0x7f4a600a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-25T20:57:37.581 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.580+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 --> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f4a74062590 con 0x7f4a600a3ca0 2024-04-25T20:57:37.613 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:37.305+0000 7effd67c0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:57:37.622 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:37.367+0000 7f6142174200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:57:37.687 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:37.379+0000 7f85d3b39200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T20:57:37.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.961+0000 7f4a79bde640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 msgr2=0x7f4a600a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.96:6800/23858312 2024-04-25T20:57:37.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:37.961+0000 7f4a79bde640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 0x7f4a600a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-25T20:57:37.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:37.686+0000 7f85d3b39200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T20:57:38.246 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:37.904+0000 7effd67c0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:57:38.247 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y[34073]: 2024-04-25T20:57:38.117+0000 7effd67c0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:57:38.247 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:37.966+0000 7f6142174200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:57:38.247 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z[34884]: 2024-04-25T20:57:38.181+0000 7f6142174200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:57:38.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:38.258+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f4a64099330 con 0x7f4a74100820 2024-04-25T20:57:38.499 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.294+0000 7f85d3b39200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/4291990036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: Standby manager daemon y restarted 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: Standby manager daemon y started 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:57:38.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: Standby manager daemon z restarted 2024-04-25T20:57:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: Standby manager daemon z started 2024-04-25T20:57:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:57:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:57:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:57:38.501 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-mon[29288]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:57:38.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:38.529+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f4a640035f0 con 0x7f4a74100820 2024-04-25T20:57:38.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:38.529+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 msgr2=0x7f4a600a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T20:57:38.530 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:38.529+0000 7f4a6b7fe640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/23858312,v1:172.21.15.96:6801/23858312] conn(0x7f4a600a3ca0 0x7f4a600a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/4291990036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: Standby manager daemon y restarted 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: Standby manager daemon y started 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: Standby manager daemon z restarted 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: Standby manager daemon z started 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:57:38.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:57:38.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:38 smithi097 ceph-mon[32667]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/4291990036' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: Standby manager daemon y restarted 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: Standby manager daemon y started 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.97:0/309801361' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: Standby manager daemon z restarted 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: Standby manager daemon z started 2024-04-25T20:57:38.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-25T20:57:38.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-25T20:57:38.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-25T20:57:38.624 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:38 smithi139 ceph-mon[33027]: from='mgr.? 172.21.15.139:0/2731997371' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-25T20:57:38.766 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.513+0000 7f85d3b39200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T20:57:39.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f8535c72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.249 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f8535c72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f8535c72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f8535c72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f8535c72700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f853346d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f853346d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f853346d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f853346d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.250 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:38.764+0000 7f853346d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.532+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mon.1 v2:172.21.15.139:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f4a640a53f0 con 0x7f4a74100820 2024-04-25T20:57:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.533+0000 7f4a6b7fe640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4a60035660 0x7f4a60035a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.534 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.533+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f4a6401ce70 con 0x7f4a60035660 2024-04-25T20:57:39.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.536+0000 7f4a79bde640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4a60035660 0x7f4a60035a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.538 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.537+0000 7f4a79bde640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4a60035660 0x7f4a60035a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a70000c00 tx=0x7f4a700314a0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: Active manager daemon x restarted 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: Activating manager daemon x 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: mgrmap e21: x(active, starting, since 0.0144964s), standbys: y, z 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:57:39.548 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:57:39.549 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:57:39.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: Manager daemon x is now available 2024-04-25T20:57:39.551 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:39.551 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:57:39.551 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:57:39.560 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.559+0000 7f4a6b7fe640 1 -- 172.21.15.96:0/3924665206 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f4a6401ce70 con 0x7f4a60035660 2024-04-25T20:57:39.560 INFO:teuthology.orchestra.run.smithi096.stdout:{} 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4a60035660 msgr2=0x7f4a60035a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4a60035660 0x7f4a60035a40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a70000c00 tx=0x7f4a700314a0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a74100820 msgr2=0x7f4a7419a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 --2- 172.21.15.96:0/3924665206 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4a74100820 0x7f4a7419a330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4a6400e9f0 tx=0x7f4a6400eeb0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7abe0640 1 -- 172.21.15.96:0/3924665206 reap_dead start 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 shutdown_connections 2024-04-25T20:57:39.562 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 >> 172.21.15.96:0/3924665206 conn(0x7f4a740fc420 msgr2=0x7f4a740fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:39.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 shutdown_connections 2024-04-25T20:57:39.563 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.561+0000 7f4a7c643640 1 -- 172.21.15.96:0/3924665206 wait complete. 2024-04-25T20:57:39.573 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-25T20:57:39.591 INFO:teuthology.run_tasks:Running task exec... 2024-04-25T20:57:39.600 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T20:57:39.600 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:57:39.600 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-25T20:57:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T20:57:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: Active manager daemon x restarted 2024-04-25T20:57:39.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: Activating manager daemon x 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: mgrmap e21: x(active, starting, since 0.0144964s), standbys: y, z 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:57:39.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:57:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: Manager daemon x is now available 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:57:39.616 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:39 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:57:39.625 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: Active manager daemon x restarted 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: Activating manager daemon x 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: osdmap e94: 12 total, 12 up, 12 in 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: mgrmap e21: x(active, starting, since 0.0144964s), standbys: y, z 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-25T20:57:39.626 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-25T20:57:39.627 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: Manager daemon x is now available 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-25T20:57:39.628 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:39 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-25T20:57:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 -- 172.21.15.96:0/2223369932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 msgr2=0x7fc5a4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 --2- 172.21.15.96:0/2223369932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a4109320 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc59c009900 tx=0x7fc59c02a410 comp rx=0 tx=0).stop 2024-04-25T20:57:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 -- 172.21.15.96:0/2223369932 shutdown_connections 2024-04-25T20:57:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 --2- 172.21.15.96:0/2223369932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a4109320 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 --2- 172.21.15.96:0/2223369932 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4102040 0x7fc5a41024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 --2- 172.21.15.96:0/2223369932 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a4100e60 0x7fc5a4101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 -- 172.21.15.96:0/2223369932 >> 172.21.15.96:0/2223369932 conn(0x7fc5a40fc420 msgr2=0x7fc5a40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 -- 172.21.15.96:0/2223369932 shutdown_connections 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.718+0000 7fc5aa144640 1 -- 172.21.15.96:0/2223369932 wait complete. 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 Processor -- start 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 -- start start 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a4100e60 0x7fc5a410b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4102040 0x7fc5a4109ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a410a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410daf0 con 0x7fc5a4102a00 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410d970 con 0x7fc5a4102040 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5aa144640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc5a410dc70 con 0x7fc5a4100e60 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a410a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a410a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52492/0 (socket says 172.21.15.96:52492) 2024-04-25T20:57:39.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5a3fff640 1 -- 172.21.15.96:0/583161847 learned_addr learned my addr 172.21.15.96:0/583161847 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5a37fe640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a4100e60 0x7fc5a410b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.719+0000 7fc5a2ffd640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4102040 0x7fc5a4109ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a3fff640 1 -- 172.21.15.96:0/583161847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a4100e60 msgr2=0x7fc5a410b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a3fff640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a4100e60 0x7fc5a410b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a3fff640 1 -- 172.21.15.96:0/583161847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4102040 msgr2=0x7fc5a4109ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a3fff640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4102040 0x7fc5a4109ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a3fff640 1 -- 172.21.15.96:0/583161847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5a410ac70 con 0x7fc5a4102a00 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a3fff640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a410a3e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc59c031500 tx=0x7fc59c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c03f030 con 0x7fc5a4102a00 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc59c003cb0 con 0x7fc5a4102a00 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5a41a2160 con 0x7fc5a4102a00 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc59c035070 con 0x7fc5a4102a00 2024-04-25T20:57:39.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.720+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5a41a26b0 con 0x7fc5a4102a00 2024-04-25T20:57:39.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.721+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5a41081a0 con 0x7fc5a4102a00 2024-04-25T20:57:39.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.722+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fc59c042030 con 0x7fc5a4102a00 2024-04-25T20:57:39.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.722+0000 7fc5a0ff9640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc5900a3a60 0x7fc5900a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:39.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.723+0000 7fc5a37fe640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc5900a3a60 0x7fc5900a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:39.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.723+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fc59c0f05c0 con 0x7fc5a4102a00 2024-04-25T20:57:39.725 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.723+0000 7fc5a37fe640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc5900a3a60 0x7fc5900a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4101bb0 tx=0x7fc594007600 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:39.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.727+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc59c0bcb90 con 0x7fc5a4102a00 2024-04-25T20:57:39.935 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:39.935+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fc5a4062390 con 0x7fc5900a3a60 2024-04-25T20:57:39.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:39.546+0000 7f8530467700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:39.546+0000 7f8530467700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:39.546+0000 7f8530467700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:39.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:39.546+0000 7f8530467700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:40.000 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:57:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:57:39.546+0000 7f8530467700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:57:40.004 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.003+0000 7fc5a0ff9640 1 -- 172.21.15.96:0/583161847 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fc5a4062390 con 0x7fc5900a3a60 2024-04-25T20:57:40.004 INFO:teuthology.orchestra.run.smithi096.stdout:Schedule set for path / 2024-04-25T20:57:40.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc5900a3a60 msgr2=0x7fc5900a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc5900a3a60 0x7fc5900a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4101bb0 tx=0x7fc594007600 comp rx=0 tx=0).stop 2024-04-25T20:57:40.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 msgr2=0x7fc5a410a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.007 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a410a3e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc59c031500 tx=0x7fc59c0042f0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 shutdown_connections 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc5900a3a60 0x7fc5900a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5a4102a00 0x7fc5a410a3e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc5a4102040 0x7fc5a4109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 --2- 172.21.15.96:0/583161847 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc5a4100e60 0x7fc5a410b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 >> 172.21.15.96:0/583161847 conn(0x7fc5a40fc420 msgr2=0x7fc5a4105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 shutdown_connections 2024-04-25T20:57:40.008 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.006+0000 7fc5aa144640 1 -- 172.21.15.96:0/583161847 wait complete. 2024-04-25T20:57:40.019 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- 172.21.15.96:0/2467267062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81031d0 msgr2=0x7fcda8103650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/2467267062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81031d0 0x7fcda8103650 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fcd94009900 tx=0x7fcd9402a410 comp rx=0 tx=0).stop 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- 172.21.15.96:0/2467267062 shutdown_connections 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/2467267062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcda8103b90 0x7fcda810a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/2467267062 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda81031d0 0x7fcda8103650 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/2467267062 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcda8101fe0 0x7fcda8102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- 172.21.15.96:0/2467267062 >> 172.21.15.96:0/2467267062 conn(0x7fcda80fd570 msgr2=0x7fcda80ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- 172.21.15.96:0/2467267062 shutdown_connections 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- 172.21.15.96:0/2467267062 wait complete. 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 Processor -- start 2024-04-25T20:57:40.131 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- start start 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcda8101fe0 0x7fcda8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcda81031d0 0x7fcda8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 0x7fcda819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810db20 con 0x7fcda8103b90 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810d9a0 con 0x7fcda81031d0 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.128+0000 7fcdaefa2640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcda810dca0 con 0x7fcda8101fe0 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 0x7fcda819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 0x7fcda819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52514/0 (socket says 172.21.15.96:52514) 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 -- 172.21.15.96:0/3892297752 learned_addr learned my addr 172.21.15.96:0/3892297752 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcda7fff640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcda81031d0 0x7fcda8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdacd3e640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcda8101fe0 0x7fcda8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 -- 172.21.15.96:0/3892297752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcda8101fe0 msgr2=0x7fcda8198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcda8101fe0 0x7fcda8198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 -- 172.21.15.96:0/3892297752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcda81031d0 msgr2=0x7fcda8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcda81031d0 0x7fcda8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 -- 172.21.15.96:0/3892297752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda819fd50 con 0x7fcda8103b90 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdad53f640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 0x7fcda819f6e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcda000eb10 tx=0x7fcda000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0004b20 con 0x7fcda8103b90 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcda0004cc0 con 0x7fcda8103b90 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda819ffe0 con 0x7fcda8103b90 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0010670 con 0x7fcda8103b90 2024-04-25T20:57:40.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.129+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda81a0500 con 0x7fcda8103b90 2024-04-25T20:57:40.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.130+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda8109350 con 0x7fcda8103b90 2024-04-25T20:57:40.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.131+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fcda0005020 con 0x7fcda8103b90 2024-04-25T20:57:40.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.132+0000 7fcda5ffb640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd980a3ae0 0x7fcd980a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.132+0000 7fcdacd3e640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd980a3ae0 0x7fcd980a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.132+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcda00d0640 con 0x7fcda8103b90 2024-04-25T20:57:40.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.132+0000 7fcdacd3e640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd980a3ae0 0x7fcd980a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcda8103030 tx=0x7fcd9c005f90 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.136+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcda009d260 con 0x7fcda8103b90 2024-04-25T20:57:40.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.339+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fcda8062430 con 0x7fcd980a3ae0 2024-04-25T20:57:40.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.399+0000 7fcda5ffb640 1 -- 172.21.15.96:0/3892297752 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fcda8062430 con 0x7fcd980a3ae0 2024-04-25T20:57:40.400 INFO:teuthology.orchestra.run.smithi096.stdout:Retention added to path / 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.401+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd980a3ae0 msgr2=0x7fcd980a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.401+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd980a3ae0 0x7fcd980a5fa0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fcda8103030 tx=0x7fcd9c005f90 comp rx=0 tx=0).stop 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.401+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 msgr2=0x7fcda819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.401+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 0x7fcda819f6e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fcda000eb10 tx=0x7fcda000c2d0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.401+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 shutdown_connections 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd980a3ae0 0x7fcd980a5fa0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcda8103b90 0x7fcda819f6e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcda81031d0 0x7fcda8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 --2- 172.21.15.96:0/3892297752 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcda8101fe0 0x7fcda8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 >> 172.21.15.96:0/3892297752 conn(0x7fcda80fd570 msgr2=0x7fcda8106520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 shutdown_connections 2024-04-25T20:57:40.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.402+0000 7fcdaefa2640 1 -- 172.21.15.96:0/3892297752 wait complete. 2024-04-25T20:57:40.413 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 -- 172.21.15.96:0/2669615346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 msgr2=0x7f0c98104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/2669615346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98104ed0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009900 tx=0x7f0c8802a410 comp rx=0 tx=0).stop 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 -- 172.21.15.96:0/2669615346 shutdown_connections 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/2669615346 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 0x7f0c9810a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/2669615346 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 0x7f0c98107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/2669615346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98104ed0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.519 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 -- 172.21.15.96:0/2669615346 >> 172.21.15.96:0/2669615346 conn(0x7f0c980fc4d0 msgr2=0x7f0c980fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 -- 172.21.15.96:0/2669615346 shutdown_connections 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.516+0000 7f0c9cb08640 1 -- 172.21.15.96:0/2669615346 wait complete. 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 Processor -- start 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 -- start start 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 0x7f0c98198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 0x7f0c9819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9810db30 con 0x7f0c98102ae0 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9810d9b0 con 0x7f0c98105410 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9cb08640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9810dcb0 con 0x7f0c98107d40 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52536/0 (socket says 172.21.15.96:52536) 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c9659c640 1 -- 172.21.15.96:0/729020377 learned_addr learned my addr 172.21.15.96:0/729020377 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:40.520 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c8dd9b640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 0x7f0c98198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.517+0000 7f0c96d9d640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 0x7f0c9819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9659c640 1 -- 172.21.15.96:0/729020377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 msgr2=0x7f0c9819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9659c640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 0x7f0c9819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9659c640 1 -- 172.21.15.96:0/729020377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 msgr2=0x7f0c98198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9659c640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 0x7f0c98198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9659c640 1 -- 172.21.15.96:0/729020377 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9819fd70 con 0x7f0c98102ae0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c96d9d640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 0x7f0c9819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c8dd9b640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 0x7f0c98198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9659c640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98198150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005940 tx=0x7f0c88004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8803f030 con 0x7f0c98102ae0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c88003cb0 con 0x7f0c98102ae0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c981a0000 con 0x7f0c98102ae0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88035070 con 0x7f0c98102ae0 2024-04-25T20:57:40.521 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.518+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c981a04f0 con 0x7f0c98102ae0 2024-04-25T20:57:40.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.519+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c980fdfc0 con 0x7f0c98102ae0 2024-04-25T20:57:40.522 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.520+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f0c88042030 con 0x7f0c98102ae0 2024-04-25T20:57:40.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.520+0000 7f0c8ffff640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0c840a3a60 0x7f0c840a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.521+0000 7f0c8dd9b640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0c840a3a60 0x7f0c840a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.521+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f0c880f0a00 con 0x7f0c98102ae0 2024-04-25T20:57:40.523 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.521+0000 7f0c8dd9b640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0c840a3a60 0x7f0c840a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c98199660 tx=0x7f0c8000b410 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.526 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.525+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c88003e60 con 0x7f0c98102ae0 2024-04-25T20:57:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:40 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:40 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:40 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:40 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:40 smithi097 ceph-mon[32667]: mgrmap e22: x(active, since 1.02056s), standbys: y, z 2024-04-25T20:57:40.733 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.732+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f0c98062390 con 0x7f0c840a3a60 2024-04-25T20:57:40.739 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.739+0000 7f0c8ffff640 1 -- 172.21.15.96:0/729020377 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f0c98062390 con 0x7f0c840a3a60 2024-04-25T20:57:40.740 INFO:teuthology.orchestra.run.smithi096.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T20:57:39", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0c840a3a60 msgr2=0x7f0c840a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0c840a3a60 0x7f0c840a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0c98199660 tx=0x7f0c8000b410 comp rx=0 tx=0).stop 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 msgr2=0x7f0c98198150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98198150 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f0c88005940 tx=0x7f0c88004430 comp rx=0 tx=0).stop 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 shutdown_connections 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0c840a3a60 0x7f0c840a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0c98107d40 0x7f0c9819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0c98105410 0x7f0c98198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.742 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 --2- 172.21.15.96:0/729020377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c98102ae0 0x7f0c98198150 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 >> 172.21.15.96:0/729020377 conn(0x7f0c980fc4d0 msgr2=0x7f0c98103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 shutdown_connections 2024-04-25T20:57:40.743 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.741+0000 7f0c9cb08640 1 -- 172.21.15.96:0/729020377 wait complete. 2024-04-25T20:57:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:40 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:40 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:40 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:40 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:40 smithi096 ceph-mon[29288]: mgrmap e22: x(active, since 1.02056s), standbys: y, z 2024-04-25T20:57:40.752 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 -- 172.21.15.96:0/1911530590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 msgr2=0x7f1ff8109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/1911530590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8109340 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0009ca0 tx=0x7f1ff002a8b0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 -- 172.21.15.96:0/1911530590 shutdown_connections 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/1911530590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8109340 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/1911530590 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 0x7f1ff8101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/1911530590 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff8102520 0x7f1ff8102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 -- 172.21.15.96:0/1911530590 >> 172.21.15.96:0/1911530590 conn(0x7f1ff80fbbb0 msgr2=0x7f1ff80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 -- 172.21.15.96:0/1911530590 shutdown_connections 2024-04-25T20:57:40.861 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.860+0000 7f1ffdc91640 1 -- 172.21.15.96:0/1911530590 wait complete. 2024-04-25T20:57:40.863 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 Processor -- start 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 -- start start 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 0x7f1ff8197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff8102520 0x7f1ff819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff810c9d0 con 0x7f1ff8101aa0 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff810c850 con 0x7f1ff8102520 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ffdc91640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ff810cb50 con 0x7f1ff81010e0 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8197580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52570/0 (socket says 172.21.15.96:52570) 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 -- 172.21.15.96:0/3884755028 learned_addr learned my addr 172.21.15.96:0/3884755028 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff77fe640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 0x7f1ff8197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff7fff640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff8102520 0x7f1ff819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 -- 172.21.15.96:0/3884755028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 msgr2=0x7f1ff8197040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.864 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 0x7f1ff8197040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 -- 172.21.15.96:0/3884755028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff8102520 msgr2=0x7f1ff819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.861+0000 7f1ff6ffd640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff8102520 0x7f1ff819e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ff6ffd640 1 -- 172.21.15.96:0/3884755028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ff819ec10 con 0x7f1ff8101aa0 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ff77fe640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 0x7f1ff8197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ff6ffd640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8197580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800e9c0 tx=0x7f1fe800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe800cd80 con 0x7f1ff8101aa0 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fe80035f0 con 0x7f1ff8101aa0 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8010690 con 0x7f1ff8101aa0 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ff819ef00 con 0x7f1ff8101aa0 2024-04-25T20:57:40.865 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.862+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ff819f410 con 0x7f1ff8101aa0 2024-04-25T20:57:40.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.863+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ff81082f0 con 0x7f1ff8101aa0 2024-04-25T20:57:40.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.864+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f1fe80037a0 con 0x7f1ff8101aa0 2024-04-25T20:57:40.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.865+0000 7f1ff4ff9640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1fe00a3ae0 0x7f1fe00a5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:40.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.865+0000 7f1ff77fe640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1fe00a3ae0 0x7f1fe00a5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:40.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.865+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f1fe80d0390 con 0x7f1ff8101aa0 2024-04-25T20:57:40.866 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.865+0000 7f1ff77fe640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1fe00a3ae0 0x7f1fe00a5fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009fc0 tx=0x7f1fec005f90 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:40.871 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:40.870+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fe809cfb0 con 0x7f1ff8101aa0 2024-04-25T20:57:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:40 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:40 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:40 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:40 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:40.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:40 smithi139 ceph-mon[33027]: mgrmap e22: x(active, since 1.02056s), standbys: y, z 2024-04-25T20:57:41.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.117+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1ff8062390 con 0x7f1fe00a3ae0 2024-04-25T20:57:41.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.124+0000 7f1ff4ff9640 1 -- 172.21.15.96:0/3884755028 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1ff8062390 con 0x7f1fe00a3ae0 2024-04-25T20:57:41.125 INFO:teuthology.orchestra.run.smithi096.stdout:/ 1m 6m3h 2024-04-25T20:57:41.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1fe00a3ae0 msgr2=0x7f1fe00a5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1fe00a3ae0 0x7f1fe00a5fa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f1fec009fc0 tx=0x7f1fec005f90 comp rx=0 tx=0).stop 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 msgr2=0x7f1ff8197580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8197580 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800e9c0 tx=0x7f1fe800ee80 comp rx=0 tx=0).stop 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 shutdown_connections 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1fe00a3ae0 0x7f1fe00a5fa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ff8102520 0x7f1ff819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ff8101aa0 0x7f1ff8197580 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 --2- 172.21.15.96:0/3884755028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ff81010e0 0x7f1ff8197040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 >> 172.21.15.96:0/3884755028 conn(0x7f1ff80fbbb0 msgr2=0x7f1ff81054c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 shutdown_connections 2024-04-25T20:57:41.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:41.126+0000 7f1ffdc91640 1 -- 172.21.15.96:0/3884755028 wait complete. 2024-04-25T20:57:41.139 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-25T20:57:41.166 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-25T20:57:41.177 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-25T20:57:41.186 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T20:57:41.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-25T20:57:41.210 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-25T20:57:41.225 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-25T20:57:41.225 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:57:41.486 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:41.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: [25/Apr/2024:20:57:39] ENGINE Bus STARTING 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: [25/Apr/2024:20:57:39] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: [25/Apr/2024:20:57:39] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: [25/Apr/2024:20:57:39] ENGINE Bus STARTED 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:41 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: [25/Apr/2024:20:57:39] ENGINE Bus STARTING 2024-04-25T20:57:41.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: [25/Apr/2024:20:57:39] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: [25/Apr/2024:20:57:39] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: [25/Apr/2024:20:57:39] ENGINE Bus STARTED 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:41 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: [25/Apr/2024:20:57:39] ENGINE Bus STARTING 2024-04-25T20:57:41.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: [25/Apr/2024:20:57:39] ENGINE Serving on https://172.21.15.96:7150 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: [25/Apr/2024:20:57:39] ENGINE Serving on http://172.21.15.96:8765 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: [25/Apr/2024:20:57:39] ENGINE Bus STARTED 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='client.15225 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='client.15231 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:41.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:41 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:42 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:42 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='client.15249 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:42.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:42 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi139 to 176.2M 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: Unable to set osd_memory_target on smithi139 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: Unable to set osd_memory_target on smithi097 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.453 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:43.454 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:43 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:43.464 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 -- 172.21.15.96:0/620912289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 msgr2=0x7fa740111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 --2- 172.21.15.96:0/620912289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa740111b90 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa73800b0d0 tx=0x7fa73800b3e0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 -- 172.21.15.96:0/620912289 shutdown_connections 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 --2- 172.21.15.96:0/620912289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa740111b90 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 --2- 172.21.15.96:0/620912289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa7401052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 --2- 172.21.15.96:0/620912289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 0x7fa7401098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 -- 172.21.15.96:0/620912289 >> 172.21.15.96:0/620912289 conn(0x7fa740100770 msgr2=0x7fa740102b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 -- 172.21.15.96:0/620912289 shutdown_connections 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.451+0000 7fa74825f700 1 -- 172.21.15.96:0/620912289 wait complete. 2024-04-25T20:57:43.465 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 Processor -- start 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 -- start start 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa74019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 0x7fa74019b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa74019fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa740114ea0 con 0x7fa740109fc0 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa740114d20 con 0x7fa740104ee0 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.452+0000 7fa74825f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa740115020 con 0x7fa740105890 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa74019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa7467fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa74019fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa7457fa700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 0x7fa74019b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.466 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa74019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:40510/0 (socket says 172.21.15.96:40510) 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 -- 172.21.15.96:0/2924510658 learned_addr learned my addr 172.21.15.96:0/2924510658 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 -- 172.21.15.96:0/2924510658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 msgr2=0x7fa74019b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 0x7fa74019b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 -- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 msgr2=0x7fa74019fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa74019fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 -- 172.21.15.96:0/2924510658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7401a02c0 con 0x7fa740104ee0 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa7457fa700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 0x7fa74019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:43.467 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa7467fc700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa74019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa745ffb700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa74019b2b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00e9a0 tx=0x7fa73c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00ca10 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.453+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7401a05b0 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.454+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7401a7e30 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.454+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa73c00cbb0 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.454+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73c018680 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.455+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa740069730 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.456+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 23) ==== 138817+0+0 (secure 0 0 0) 0x7fa73c010430 con 0x7fa740104ee0 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.456+0000 7fa736ffd700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa72c0a3ec0 0x7fa72c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:43.468 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.456+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fa73c0d0aa0 con 0x7fa740104ee0 2024-04-25T20:57:43.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.456+0000 7fa7457fa700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa72c0a3ec0 0x7fa72c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:43.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.457+0000 7fa7457fa700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa72c0a3ec0 0x7fa72c0a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa74010b4a0 tx=0x7fa730009260 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:43.469 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.461+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa73c09d040 con 0x7fa740104ee0 2024-04-25T20:57:43.500 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.498+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fa73c098fb0 con 0x7fa740104ee0 2024-04-25T20:57:43.686 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.685+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa7401018f0 con 0x7fa72c0a3ec0 2024-04-25T20:57:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.693+0000 7fa736ffd700 1 -- 172.21.15.96:0/2924510658 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257741 (secure 0 0 0) 0x7fa7401018f0 con 0x7fa72c0a3ec0 2024-04-25T20:57:43.694 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa72c0a3ec0 msgr2=0x7fa72c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa72c0a3ec0 0x7fa72c0a6380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa74010b4a0 tx=0x7fa730009260 comp rx=0 tx=0).stop 2024-04-25T20:57:43.700 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 msgr2=0x7fa74019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa74019b2b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa73c00e9a0 tx=0x7fa73c00ecb0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 shutdown_connections 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa72c0a3ec0 0x7fa72c0a6380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa740109fc0 0x7fa74019fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa740104ee0 0x7fa74019b2b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 --2- 172.21.15.96:0/2924510658 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa740105890 0x7fa74019b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 >> 172.21.15.96:0/2924510658 conn(0x7fa740100770 msgr2=0x7fa7401011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 shutdown_connections 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:43.697+0000 7fa74825f700 1 -- 172.21.15.96:0/2924510658 wait complete. 2024-04-25T20:57:43.701 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi139 to 176.2M 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: Unable to set osd_memory_target on smithi139 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: Unable to set osd_memory_target on smithi097 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:43.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:43 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:43.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi139 to 176.2M 2024-04-25T20:57:43.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: Unable to set osd_memory_target on smithi139 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T20:57:43.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: Unable to set osd_memory_target on smithi097 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:43.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:43 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:44.408 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-25T20:57:42.535874+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187318,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1298,"num_read_kb":808,"num_write":476,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389928,"kb_used_data":15728,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551896,"statfs":{"total":1151940427776,"available":1151541141504,"internally_reserved":0,"allocated":16105472,"data_stored":9594478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.987381"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191422+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:57:39.191422+0000","last_peered":"2024-04-25T20:57:39.191422+0000","last_clean":"2024-04-25T20:57:39.191422+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:57:39.191422+0000","last_undegraded":"2024-04-25T20:57:39.191422+0000","last_fullsized":"2024-04-25T20:57:39.191422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691744+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:57:38.691744+0000","last_peered":"2024-04-25T20:57:38.691744+0000","last_clean":"2024-04-25T20:57:38.691744+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:57:38.691744+0000","last_undegraded":"2024-04-25T20:57:38.691744+0000","last_fullsized":"2024-04-25T20:57:38.691744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691501+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:57:38.691501+0000","last_peered":"2024-04-25T20:57:38.691501+0000","last_clean":"2024-04-25T20:57:38.691501+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:57:38.691501+0000","last_undegraded":"2024-04-25T20:57:38.691501+0000","last_fullsized":"2024-04-25T20:57:38.691501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616317+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:57:38.616317+0000","last_peered":"2024-04-25T20:57:38.616317+0000","last_clean":"2024-04-25T20:57:38.616317+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:57:38.616317+0000","last_undegraded":"2024-04-25T20:57:38.616317+0000","last_fullsized":"2024-04-25T20:57:38.616317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622554+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:57:38.622554+0000","last_peered":"2024-04-25T20:57:38.622554+0000","last_clean":"2024-04-25T20:57:38.622554+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:57:38.622554+0000","last_undegraded":"2024-04-25T20:57:38.622554+0000","last_fullsized":"2024-04-25T20:57:38.622554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522256+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:57:38.522256+0000","last_peered":"2024-04-25T20:57:38.522256+0000","last_clean":"2024-04-25T20:57:38.522256+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:57:38.522256+0000","last_undegraded":"2024-04-25T20:57:38.522256+0000","last_fullsized":"2024-04-25T20:57:38.522256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691132+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:57:38.691132+0000","last_peered":"2024-04-25T20:57:38.691132+0000","last_clean":"2024-04-25T20:57:38.691132+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:57:38.691132+0000","last_undegraded":"2024-04-25T20:57:38.691132+0000","last_fullsized":"2024-04-25T20:57:38.691132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523035+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:57:38.523035+0000","last_peered":"2024-04-25T20:57:38.523035+0000","last_clean":"2024-04-25T20:57:38.523035+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:57:38.523035+0000","last_undegraded":"2024-04-25T20:57:38.523035+0000","last_fullsized":"2024-04-25T20:57:38.523035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692124+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:57:38.692124+0000","last_peered":"2024-04-25T20:57:38.692124+0000","last_clean":"2024-04-25T20:57:38.692124+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:57:38.692124+0000","last_undegraded":"2024-04-25T20:57:38.692124+0000","last_fullsized":"2024-04-25T20:57:38.692124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522625+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:57:38.522625+0000","last_peered":"2024-04-25T20:57:38.522625+0000","last_clean":"2024-04-25T20:57:38.522625+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:57:38.522625+0000","last_undegraded":"2024-04-25T20:57:38.522625+0000","last_fullsized":"2024-04-25T20:57:38.522625+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692177+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:57:38.692177+0000","last_peered":"2024-04-25T20:57:38.692177+0000","last_clean":"2024-04-25T20:57:38.692177+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:57:38.692177+0000","last_undegraded":"2024-04-25T20:57:38.692177+0000","last_fullsized":"2024-04-25T20:57:38.692177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616238+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:57:38.616238+0000","last_peered":"2024-04-25T20:57:38.616238+0000","last_clean":"2024-04-25T20:57:38.616238+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:57:38.616238+0000","last_undegraded":"2024-04-25T20:57:38.616238+0000","last_fullsized":"2024-04-25T20:57:38.616238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614743+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:57:38.614743+0000","last_peered":"2024-04-25T20:57:38.614743+0000","last_clean":"2024-04-25T20:57:38.614743+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:57:38.614743+0000","last_undegraded":"2024-04-25T20:57:38.614743+0000","last_fullsized":"2024-04-25T20:57:38.614743+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691157+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:57:38.691157+0000","last_peered":"2024-04-25T20:57:38.691157+0000","last_clean":"2024-04-25T20:57:38.691157+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:57:38.691157+0000","last_undegraded":"2024-04-25T20:57:38.691157+0000","last_fullsized":"2024-04-25T20:57:38.691157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691059+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:57:38.691059+0000","last_peered":"2024-04-25T20:57:38.691059+0000","last_clean":"2024-04-25T20:57:38.691059+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:57:38.691059+0000","last_undegraded":"2024-04-25T20:57:38.691059+0000","last_fullsized":"2024-04-25T20:57:38.691059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691068+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:57:38.691068+0000","last_peered":"2024-04-25T20:57:38.691068+0000","last_clean":"2024-04-25T20:57:38.691068+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:57:38.691068+0000","last_undegraded":"2024-04-25T20:57:38.691068+0000","last_fullsized":"2024-04-25T20:57:38.691068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191562+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:57:39.191562+0000","last_peered":"2024-04-25T20:57:39.191562+0000","last_clean":"2024-04-25T20:57:39.191562+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:57:39.191562+0000","last_undegraded":"2024-04-25T20:57:39.191562+0000","last_fullsized":"2024-04-25T20:57:39.191562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.620534+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:57:38.620534+0000","last_peered":"2024-04-25T20:57:38.620534+0000","last_clean":"2024-04-25T20:57:38.620534+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:57:38.620534+0000","last_undegraded":"2024-04-25T20:57:38.620534+0000","last_fullsized":"2024-04-25T20:57:38.620534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524590+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:57:38.524590+0000","last_peered":"2024-04-25T20:57:38.524590+0000","last_clean":"2024-04-25T20:57:38.524590+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:57:38.524590+0000","last_undegraded":"2024-04-25T20:57:38.524590+0000","last_fullsized":"2024-04-25T20:57:38.524590+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.620316+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:57:38.620316+0000","last_peered":"2024-04-25T20:57:38.620316+0000","last_clean":"2024-04-25T20:57:38.620316+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:57:38.620316+0000","last_undegraded":"2024-04-25T20:57:38.620316+0000","last_fullsized":"2024-04-25T20:57:38.620316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523372+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:57:38.523372+0000","last_peered":"2024-04-25T20:57:38.523372+0000","last_clean":"2024-04-25T20:57:38.523372+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:57:38.523372+0000","last_undegraded":"2024-04-25T20:57:38.523372+0000","last_fullsized":"2024-04-25T20:57:38.523372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523534+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:57:38.523534+0000","last_peered":"2024-04-25T20:57:38.523534+0000","last_clean":"2024-04-25T20:57:38.523534+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:57:38.523534+0000","last_undegraded":"2024-04-25T20:57:38.523534+0000","last_fullsized":"2024-04-25T20:57:38.523534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691859+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:57:38.691859+0000","last_peered":"2024-04-25T20:57:38.691859+0000","last_clean":"2024-04-25T20:57:38.691859+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:57:38.691859+0000","last_undegraded":"2024-04-25T20:57:38.691859+0000","last_fullsized":"2024-04-25T20:57:38.691859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622881+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:57:38.622881+0000","last_peered":"2024-04-25T20:57:38.622881+0000","last_clean":"2024-04-25T20:57:38.622881+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:57:38.622881+0000","last_undegraded":"2024-04-25T20:57:38.622881+0000","last_fullsized":"2024-04-25T20:57:38.622881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691786+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:57:38.691786+0000","last_peered":"2024-04-25T20:57:38.691786+0000","last_clean":"2024-04-25T20:57:38.691786+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:57:38.691786+0000","last_undegraded":"2024-04-25T20:57:38.691786+0000","last_fullsized":"2024-04-25T20:57:38.691786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622783+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:57:38.622783+0000","last_peered":"2024-04-25T20:57:38.622783+0000","last_clean":"2024-04-25T20:57:38.622783+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:57:38.622783+0000","last_undegraded":"2024-04-25T20:57:38.622783+0000","last_fullsized":"2024-04-25T20:57:38.622783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616254+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:57:38.616254+0000","last_peered":"2024-04-25T20:57:38.616254+0000","last_clean":"2024-04-25T20:57:38.616254+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:57:38.616254+0000","last_undegraded":"2024-04-25T20:57:38.616254+0000","last_fullsized":"2024-04-25T20:57:38.616254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":216,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616381+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:57:38.616381+0000","last_peered":"2024-04-25T20:57:38.616381+0000","last_clean":"2024-04-25T20:57:38.616381+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:57:38.616381+0000","last_undegraded":"2024-04-25T20:57:38.616381+0000","last_fullsized":"2024-04-25T20:57:38.616381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622512+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:57:38.622512+0000","last_peered":"2024-04-25T20:57:38.622512+0000","last_clean":"2024-04-25T20:57:38.622512+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:57:38.622512+0000","last_undegraded":"2024-04-25T20:57:38.622512+0000","last_fullsized":"2024-04-25T20:57:38.622512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522207+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:57:38.522207+0000","last_peered":"2024-04-25T20:57:38.522207+0000","last_clean":"2024-04-25T20:57:38.522207+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:57:38.522207+0000","last_undegraded":"2024-04-25T20:57:38.522207+0000","last_fullsized":"2024-04-25T20:57:38.522207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615952+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:57:38.615952+0000","last_peered":"2024-04-25T20:57:38.615952+0000","last_clean":"2024-04-25T20:57:38.615952+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:57:38.615952+0000","last_undegraded":"2024-04-25T20:57:38.615952+0000","last_fullsized":"2024-04-25T20:57:38.615952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614503+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:57:38.614503+0000","last_peered":"2024-04-25T20:57:38.614503+0000","last_clean":"2024-04-25T20:57:38.614503+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:57:38.614503+0000","last_undegraded":"2024-04-25T20:57:38.614503+0000","last_fullsized":"2024-04-25T20:57:38.614503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522962+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:57:38.522962+0000","last_peered":"2024-04-25T20:57:38.522962+0000","last_clean":"2024-04-25T20:57:38.522962+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:57:38.522962+0000","last_undegraded":"2024-04-25T20:57:38.522962+0000","last_fullsized":"2024-04-25T20:57:38.522962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615783+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:57:38.615783+0000","last_peered":"2024-04-25T20:57:38.615783+0000","last_clean":"2024-04-25T20:57:38.615783+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:57:38.615783+0000","last_undegraded":"2024-04-25T20:57:38.615783+0000","last_fullsized":"2024-04-25T20:57:38.615783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.619924+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:57:38.619924+0000","last_peered":"2024-04-25T20:57:38.619924+0000","last_clean":"2024-04-25T20:57:38.619924+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:57:38.619924+0000","last_undegraded":"2024-04-25T20:57:38.619924+0000","last_fullsized":"2024-04-25T20:57:38.619924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":206,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522702+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:57:38.522702+0000","last_peered":"2024-04-25T20:57:38.522702+0000","last_clean":"2024-04-25T20:57:38.522702+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:57:38.522702+0000","last_undegraded":"2024-04-25T20:57:38.522702+0000","last_fullsized":"2024-04-25T20:57:38.522702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691858+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:57:38.691858+0000","last_peered":"2024-04-25T20:57:38.691858+0000","last_clean":"2024-04-25T20:57:38.691858+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:57:38.691858+0000","last_undegraded":"2024-04-25T20:57:38.691858+0000","last_fullsized":"2024-04-25T20:57:38.691858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692223+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:57:38.692223+0000","last_peered":"2024-04-25T20:57:38.692223+0000","last_clean":"2024-04-25T20:57:38.692223+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:57:38.692223+0000","last_undegraded":"2024-04-25T20:57:38.692223+0000","last_fullsized":"2024-04-25T20:57:38.692223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691875+0000","last_change":"2024-04-25T20:55:10.334414+0000","last_active":"2024-04-25T20:57:38.691875+0000","last_peered":"2024-04-25T20:57:38.691875+0000","last_clean":"2024-04-25T20:57:38.691875+0000","last_became_active":"2024-04-25T20:55:10.334022+0000","last_became_peered":"2024-04-25T20:55:10.334022+0000","last_unstale":"2024-04-25T20:57:38.691875+0000","last_undegraded":"2024-04-25T20:57:38.691875+0000","last_fullsized":"2024-04-25T20:57:38.691875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:32.914876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615739+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:57:38.615739+0000","last_peered":"2024-04-25T20:57:38.615739+0000","last_clean":"2024-04-25T20:57:38.615739+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:57:38.615739+0000","last_undegraded":"2024-04-25T20:57:38.615739+0000","last_fullsized":"2024-04-25T20:57:38.615739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691880+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:57:38.691880+0000","last_peered":"2024-04-25T20:57:38.691880+0000","last_clean":"2024-04-25T20:57:38.691880+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:57:38.691880+0000","last_undegraded":"2024-04-25T20:57:38.691880+0000","last_fullsized":"2024-04-25T20:57:38.691880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'37","reported_seq":423,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.769802+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:57:38.769802+0000","last_peered":"2024-04-25T20:57:38.769802+0000","last_clean":"2024-04-25T20:57:38.769802+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:57:38.769802+0000","last_undegraded":"2024-04-25T20:57:38.769802+0000","last_fullsized":"2024-04-25T20:57:38.769802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":358,"num_read_kb":244,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622687+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:57:38.622687+0000","last_peered":"2024-04-25T20:57:38.622687+0000","last_clean":"2024-04-25T20:57:38.622687+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:57:38.622687+0000","last_undegraded":"2024-04-25T20:57:38.622687+0000","last_fullsized":"2024-04-25T20:57:38.622687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524419+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:57:38.524419+0000","last_peered":"2024-04-25T20:57:38.524419+0000","last_clean":"2024-04-25T20:57:38.524419+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:57:38.524419+0000","last_undegraded":"2024-04-25T20:57:38.524419+0000","last_fullsized":"2024-04-25T20:57:38.524419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614481+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:57:38.614481+0000","last_peered":"2024-04-25T20:57:38.614481+0000","last_clean":"2024-04-25T20:57:38.614481+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:57:38.614481+0000","last_undegraded":"2024-04-25T20:57:38.614481+0000","last_fullsized":"2024-04-25T20:57:38.614481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616027+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:57:38.616027+0000","last_peered":"2024-04-25T20:57:38.616027+0000","last_clean":"2024-04-25T20:57:38.616027+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:57:38.616027+0000","last_undegraded":"2024-04-25T20:57:38.616027+0000","last_fullsized":"2024-04-25T20:57:38.616027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691679+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:57:38.691679+0000","last_peered":"2024-04-25T20:57:38.691679+0000","last_clean":"2024-04-25T20:57:38.691679+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:57:38.691679+0000","last_undegraded":"2024-04-25T20:57:38.691679+0000","last_fullsized":"2024-04-25T20:57:38.691679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":206,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616237+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:57:38.616237+0000","last_peered":"2024-04-25T20:57:38.616237+0000","last_clean":"2024-04-25T20:57:38.616237+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:57:38.616237+0000","last_undegraded":"2024-04-25T20:57:38.616237+0000","last_fullsized":"2024-04-25T20:57:38.616237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691075+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:57:38.691075+0000","last_peered":"2024-04-25T20:57:38.691075+0000","last_clean":"2024-04-25T20:57:38.691075+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:57:38.691075+0000","last_undegraded":"2024-04-25T20:57:38.691075+0000","last_fullsized":"2024-04-25T20:57:38.691075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622432+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:57:38.622432+0000","last_peered":"2024-04-25T20:57:38.622432+0000","last_clean":"2024-04-25T20:57:38.622432+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:57:38.622432+0000","last_undegraded":"2024-04-25T20:57:38.622432+0000","last_fullsized":"2024-04-25T20:57:38.622432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615853+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:57:38.615853+0000","last_peered":"2024-04-25T20:57:38.615853+0000","last_clean":"2024-04-25T20:57:38.615853+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:57:38.615853+0000","last_undegraded":"2024-04-25T20:57:38.615853+0000","last_fullsized":"2024-04-25T20:57:38.615853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622920+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:57:38.622920+0000","last_peered":"2024-04-25T20:57:38.622920+0000","last_clean":"2024-04-25T20:57:38.622920+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:57:38.622920+0000","last_undegraded":"2024-04-25T20:57:38.622920+0000","last_fullsized":"2024-04-25T20:57:38.622920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191264+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:57:39.191264+0000","last_peered":"2024-04-25T20:57:39.191264+0000","last_clean":"2024-04-25T20:57:39.191264+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:57:39.191264+0000","last_undegraded":"2024-04-25T20:57:39.191264+0000","last_fullsized":"2024-04-25T20:57:39.191264+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.619738+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:57:38.619738+0000","last_peered":"2024-04-25T20:57:38.619738+0000","last_clean":"2024-04-25T20:57:38.619738+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:57:38.619738+0000","last_undegraded":"2024-04-25T20:57:38.619738+0000","last_fullsized":"2024-04-25T20:57:38.619738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522451+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:57:38.522451+0000","last_peered":"2024-04-25T20:57:38.522451+0000","last_clean":"2024-04-25T20:57:38.522451+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:57:38.522451+0000","last_undegraded":"2024-04-25T20:57:38.522451+0000","last_fullsized":"2024-04-25T20:57:38.522451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691667+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:57:38.691667+0000","last_peered":"2024-04-25T20:57:38.691667+0000","last_clean":"2024-04-25T20:57:38.691667+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:57:38.691667+0000","last_undegraded":"2024-04-25T20:57:38.691667+0000","last_fullsized":"2024-04-25T20:57:38.691667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522266+0000","last_change":"2024-04-25T20:55:10.332432+0000","last_active":"2024-04-25T20:57:38.522266+0000","last_peered":"2024-04-25T20:57:38.522266+0000","last_clean":"2024-04-25T20:57:38.522266+0000","last_became_active":"2024-04-25T20:55:10.332033+0000","last_became_peered":"2024-04-25T20:55:10.332033+0000","last_unstale":"2024-04-25T20:57:38.522266+0000","last_undegraded":"2024-04-25T20:57:38.522266+0000","last_fullsized":"2024-04-25T20:57:38.522266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:34.930705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616167+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:57:38.616167+0000","last_peered":"2024-04-25T20:57:38.616167+0000","last_clean":"2024-04-25T20:57:38.616167+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:57:38.616167+0000","last_undegraded":"2024-04-25T20:57:38.616167+0000","last_fullsized":"2024-04-25T20:57:38.616167+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":206,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614347+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:57:38.614347+0000","last_peered":"2024-04-25T20:57:38.614347+0000","last_clean":"2024-04-25T20:57:38.614347+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:57:38.614347+0000","last_undegraded":"2024-04-25T20:57:38.614347+0000","last_fullsized":"2024-04-25T20:57:38.614347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523216+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:57:38.523216+0000","last_peered":"2024-04-25T20:57:38.523216+0000","last_clean":"2024-04-25T20:57:38.523216+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:57:38.523216+0000","last_undegraded":"2024-04-25T20:57:38.523216+0000","last_fullsized":"2024-04-25T20:57:38.523216+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616224+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:57:38.616224+0000","last_peered":"2024-04-25T20:57:38.616224+0000","last_clean":"2024-04-25T20:57:38.616224+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:57:38.616224+0000","last_undegraded":"2024-04-25T20:57:38.616224+0000","last_fullsized":"2024-04-25T20:57:38.616224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691261+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:57:38.691261+0000","last_peered":"2024-04-25T20:57:38.691261+0000","last_clean":"2024-04-25T20:57:38.691261+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:57:38.691261+0000","last_undegraded":"2024-04-25T20:57:38.691261+0000","last_fullsized":"2024-04-25T20:57:38.691261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614336+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:57:38.614336+0000","last_peered":"2024-04-25T20:57:38.614336+0000","last_clean":"2024-04-25T20:57:38.614336+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:57:38.614336+0000","last_undegraded":"2024-04-25T20:57:38.614336+0000","last_fullsized":"2024-04-25T20:57:38.614336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691380+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:57:38.691380+0000","last_peered":"2024-04-25T20:57:38.691380+0000","last_clean":"2024-04-25T20:57:38.691380+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:57:38.691380+0000","last_undegraded":"2024-04-25T20:57:38.691380+0000","last_fullsized":"2024-04-25T20:57:38.691380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191351+0000","last_change":"2024-04-25T20:55:10.337450+0000","last_active":"2024-04-25T20:57:39.191351+0000","last_peered":"2024-04-25T20:57:39.191351+0000","last_clean":"2024-04-25T20:57:39.191351+0000","last_became_active":"2024-04-25T20:55:10.332821+0000","last_became_peered":"2024-04-25T20:55:10.332821+0000","last_unstale":"2024-04-25T20:57:39.191351+0000","last_undegraded":"2024-04-25T20:57:39.191351+0000","last_fullsized":"2024-04-25T20:57:39.191351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:55.070702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614931+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:57:38.614931+0000","last_peered":"2024-04-25T20:57:38.614931+0000","last_clean":"2024-04-25T20:57:38.614931+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:57:38.614931+0000","last_undegraded":"2024-04-25T20:57:38.614931+0000","last_fullsized":"2024-04-25T20:57:38.614931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":100,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.739095+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:57:38.739095+0000","last_peered":"2024-04-25T20:57:38.739095+0000","last_clean":"2024-04-25T20:57:38.739095+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:57:38.739095+0000","last_undegraded":"2024-04-25T20:57:38.739095+0000","last_fullsized":"2024-04-25T20:57:38.739095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":87,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523457+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:57:38.523457+0000","last_peered":"2024-04-25T20:57:38.523457+0000","last_clean":"2024-04-25T20:57:38.523457+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:57:38.523457+0000","last_undegraded":"2024-04-25T20:57:38.523457+0000","last_fullsized":"2024-04-25T20:57:38.523457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616365+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:57:38.616365+0000","last_peered":"2024-04-25T20:57:38.616365+0000","last_clean":"2024-04-25T20:57:38.616365+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:57:38.616365+0000","last_undegraded":"2024-04-25T20:57:38.616365+0000","last_fullsized":"2024-04-25T20:57:38.616365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691756+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:57:38.691756+0000","last_peered":"2024-04-25T20:57:38.691756+0000","last_clean":"2024-04-25T20:57:38.691756+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:57:38.691756+0000","last_undegraded":"2024-04-25T20:57:38.691756+0000","last_fullsized":"2024-04-25T20:57:38.691756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691035+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:57:38.691035+0000","last_peered":"2024-04-25T20:57:38.691035+0000","last_clean":"2024-04-25T20:57:38.691035+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:57:38.691035+0000","last_undegraded":"2024-04-25T20:57:38.691035+0000","last_fullsized":"2024-04-25T20:57:38.691035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622483+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:57:38.622483+0000","last_peered":"2024-04-25T20:57:38.622483+0000","last_clean":"2024-04-25T20:57:38.622483+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:57:38.622483+0000","last_undegraded":"2024-04-25T20:57:38.622483+0000","last_fullsized":"2024-04-25T20:57:38.622483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523034+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:57:38.523034+0000","last_peered":"2024-04-25T20:57:38.523034+0000","last_clean":"2024-04-25T20:57:38.523034+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:57:38.523034+0000","last_undegraded":"2024-04-25T20:57:38.523034+0000","last_fullsized":"2024-04-25T20:57:38.523034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191000+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:57:39.191000+0000","last_peered":"2024-04-25T20:57:39.191000+0000","last_clean":"2024-04-25T20:57:39.191000+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:57:39.191000+0000","last_undegraded":"2024-04-25T20:57:39.191000+0000","last_fullsized":"2024-04-25T20:57:39.191000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191125+0000","last_change":"2024-04-25T20:55:10.339729+0000","last_active":"2024-04-25T20:57:39.191125+0000","last_peered":"2024-04-25T20:57:39.191125+0000","last_clean":"2024-04-25T20:57:39.191125+0000","last_became_active":"2024-04-25T20:55:10.333756+0000","last_became_peered":"2024-04-25T20:55:10.333756+0000","last_unstale":"2024-04-25T20:57:39.191125+0000","last_undegraded":"2024-04-25T20:57:39.191125+0000","last_fullsized":"2024-04-25T20:57:39.191125+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:48.806227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523332+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:57:38.523332+0000","last_peered":"2024-04-25T20:57:38.523332+0000","last_clean":"2024-04-25T20:57:38.523332+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:57:38.523332+0000","last_undegraded":"2024-04-25T20:57:38.523332+0000","last_fullsized":"2024-04-25T20:57:38.523332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622600+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:57:38.622600+0000","last_peered":"2024-04-25T20:57:38.622600+0000","last_clean":"2024-04-25T20:57:38.622600+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:57:38.622600+0000","last_undegraded":"2024-04-25T20:57:38.622600+0000","last_fullsized":"2024-04-25T20:57:38.622600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691778+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:57:38.691778+0000","last_peered":"2024-04-25T20:57:38.691778+0000","last_clean":"2024-04-25T20:57:38.691778+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:57:38.691778+0000","last_undegraded":"2024-04-25T20:57:38.691778+0000","last_fullsized":"2024-04-25T20:57:38.691778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524689+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:57:38.524689+0000","last_peered":"2024-04-25T20:57:38.524689+0000","last_clean":"2024-04-25T20:57:38.524689+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:57:38.524689+0000","last_undegraded":"2024-04-25T20:57:38.524689+0000","last_fullsized":"2024-04-25T20:57:38.524689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692261+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:57:38.692261+0000","last_peered":"2024-04-25T20:57:38.692261+0000","last_clean":"2024-04-25T20:57:38.692261+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:57:38.692261+0000","last_undegraded":"2024-04-25T20:57:38.692261+0000","last_fullsized":"2024-04-25T20:57:38.692261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'39","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.732731+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:57:38.732731+0000","last_peered":"2024-04-25T20:57:38.732731+0000","last_clean":"2024-04-25T20:57:38.732731+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:57:38.732731+0000","last_undegraded":"2024-04-25T20:57:38.732731+0000","last_fullsized":"2024-04-25T20:57:38.732731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691515+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:57:38.691515+0000","last_peered":"2024-04-25T20:57:38.691515+0000","last_clean":"2024-04-25T20:57:38.691515+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:57:38.691515+0000","last_undegraded":"2024-04-25T20:57:38.691515+0000","last_fullsized":"2024-04-25T20:57:38.691515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614981+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:57:38.614981+0000","last_peered":"2024-04-25T20:57:38.614981+0000","last_clean":"2024-04-25T20:57:38.614981+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:57:38.614981+0000","last_undegraded":"2024-04-25T20:57:38.614981+0000","last_fullsized":"2024-04-25T20:57:38.614981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692338+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:57:38.692338+0000","last_peered":"2024-04-25T20:57:38.692338+0000","last_clean":"2024-04-25T20:57:38.692338+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:57:38.692338+0000","last_undegraded":"2024-04-25T20:57:38.692338+0000","last_fullsized":"2024-04-25T20:57:38.692338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191053+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:57:39.191053+0000","last_peered":"2024-04-25T20:57:39.191053+0000","last_clean":"2024-04-25T20:57:39.191053+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:57:39.191053+0000","last_undegraded":"2024-04-25T20:57:39.191053+0000","last_fullsized":"2024-04-25T20:57:39.191053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524295+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:57:38.524295+0000","last_peered":"2024-04-25T20:57:38.524295+0000","last_clean":"2024-04-25T20:57:38.524295+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:57:38.524295+0000","last_undegraded":"2024-04-25T20:57:38.524295+0000","last_fullsized":"2024-04-25T20:57:38.524295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524391+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:57:38.524391+0000","last_peered":"2024-04-25T20:57:38.524391+0000","last_clean":"2024-04-25T20:57:38.524391+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:57:38.524391+0000","last_undegraded":"2024-04-25T20:57:38.524391+0000","last_fullsized":"2024-04-25T20:57:38.524391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691369+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:57:38.691369+0000","last_peered":"2024-04-25T20:57:38.691369+0000","last_clean":"2024-04-25T20:57:38.691369+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:57:38.691369+0000","last_undegraded":"2024-04-25T20:57:38.691369+0000","last_fullsized":"2024-04-25T20:57:38.691369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614940+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:57:38.614940+0000","last_peered":"2024-04-25T20:57:38.614940+0000","last_clean":"2024-04-25T20:57:38.614940+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:57:38.614940+0000","last_undegraded":"2024-04-25T20:57:38.614940+0000","last_fullsized":"2024-04-25T20:57:38.614940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191270+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:57:39.191270+0000","last_peered":"2024-04-25T20:57:39.191270+0000","last_clean":"2024-04-25T20:57:39.191270+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:57:39.191270+0000","last_undegraded":"2024-04-25T20:57:39.191270+0000","last_fullsized":"2024-04-25T20:57:39.191270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524541+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:57:38.524541+0000","last_peered":"2024-04-25T20:57:38.524541+0000","last_clean":"2024-04-25T20:57:38.524541+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:57:38.524541+0000","last_undegraded":"2024-04-25T20:57:38.524541+0000","last_fullsized":"2024-04-25T20:57:38.524541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523416+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:57:38.523416+0000","last_peered":"2024-04-25T20:57:38.523416+0000","last_clean":"2024-04-25T20:57:38.523416+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:57:38.523416+0000","last_undegraded":"2024-04-25T20:57:38.523416+0000","last_fullsized":"2024-04-25T20:57:38.523416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615843+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:57:38.615843+0000","last_peered":"2024-04-25T20:57:38.615843+0000","last_clean":"2024-04-25T20:57:38.615843+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:57:38.615843+0000","last_undegraded":"2024-04-25T20:57:38.615843+0000","last_fullsized":"2024-04-25T20:57:38.615843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522987+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:57:38.522987+0000","last_peered":"2024-04-25T20:57:38.522987+0000","last_clean":"2024-04-25T20:57:38.522987+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:57:38.522987+0000","last_undegraded":"2024-04-25T20:57:38.522987+0000","last_fullsized":"2024-04-25T20:57:38.522987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691387+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:57:38.691387+0000","last_peered":"2024-04-25T20:57:38.691387+0000","last_clean":"2024-04-25T20:57:38.691387+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:57:38.691387+0000","last_undegraded":"2024-04-25T20:57:38.691387+0000","last_fullsized":"2024-04-25T20:57:38.691387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614712+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:57:38.614712+0000","last_peered":"2024-04-25T20:57:38.614712+0000","last_clean":"2024-04-25T20:57:38.614712+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:57:38.614712+0000","last_undegraded":"2024-04-25T20:57:38.614712+0000","last_fullsized":"2024-04-25T20:57:38.614712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191048+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:57:39.191048+0000","last_peered":"2024-04-25T20:57:39.191048+0000","last_clean":"2024-04-25T20:57:39.191048+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:57:39.191048+0000","last_undegraded":"2024-04-25T20:57:39.191048+0000","last_fullsized":"2024-04-25T20:57:39.191048+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596716,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1162,"num_read_kb":689,"num_write":293,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1790148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":495217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547260,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37160,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707992,"statfs":{"total":95995035648,"available":95956983808,"internally_reserved":0,"allocated":1867776,"data_stored":1294204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743488,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36744,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708408,"statfs":{"total":95995035648,"available":95957409792,"internally_reserved":0,"allocated":1437696,"data_stored":897516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005124,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1433600,"data_stored":897414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1096599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":503689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":570541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":504960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":573122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1028816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":504126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724629,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28820,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716332,"statfs":{"total":95995035648,"available":95965523968,"internally_reserved":0,"allocated":1785856,"data_stored":1228274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:57:44.412 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:57:44.679 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.751 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:44.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:44.752 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:44 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:44.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:44.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:44.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:44 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi096:/etc/ceph/ceph.conf 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi097:/etc/ceph/ceph.conf 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi139:/etc/ceph/ceph.conf 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.873 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi139:/etc/ceph/ceph.client.admin.keyring 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: Updating smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.874 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:44.875 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:57:44.875 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:57:44.875 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:44 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:57:45.747 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:45 smithi096 ceph-mon[29288]: from='client.24812 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:45.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:45 smithi097 ceph-mon[32667]: from='client.24812 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:45.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:45 smithi139 ceph-mon[33027]: from='client.24812 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:46.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 -- 172.21.15.96:0/3216129195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b30102af0 msgr2=0x7f0b30102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 --2- 172.21.15.96:0/3216129195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b30102af0 0x7f0b30102f70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c009990 tx=0x7f0b2c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 -- 172.21.15.96:0/3216129195 shutdown_connections 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 --2- 172.21.15.96:0/3216129195 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b301034b0 0x7f0b3010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 --2- 172.21.15.96:0/3216129195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30108a80 0x7f0b30108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 --2- 172.21.15.96:0/3216129195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b30102af0 0x7f0b30102f70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 -- 172.21.15.96:0/3216129195 >> 172.21.15.96:0/3216129195 conn(0x7f0b30079b20 msgr2=0x7f0b30079f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 -- 172.21.15.96:0/3216129195 shutdown_connections 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.324+0000 7f0b383f9700 1 -- 172.21.15.96:0/3216129195 wait complete. 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 Processor -- start 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 -- start start 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30102af0 0x7f0b301991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 0x7f0b30199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b30108a80 0x7f0b3019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112c50 con 0x7f0b301034b0 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112ad0 con 0x7f0b30102af0 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b383f9700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0b30112dd0 con 0x7f0b30108a80 2024-04-25T20:57:46.328 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.325+0000 7f0b35994700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 0x7f0b30199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 0x7f0b30199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52610/0 (socket says 172.21.15.96:52610) 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 -- 172.21.15.96:0/4127533054 learned_addr learned my addr 172.21.15.96:0/4127533054 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b36195700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30102af0 0x7f0b301991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b36996700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b30108a80 0x7f0b3019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 -- 172.21.15.96:0/4127533054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b30108a80 msgr2=0x7f0b3019dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b30108a80 0x7f0b3019dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 -- 172.21.15.96:0/4127533054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30102af0 msgr2=0x7f0b301991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30102af0 0x7f0b301991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 -- 172.21.15.96:0/4127533054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b3019e1f0 con 0x7f0b301034b0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b36195700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30102af0 0x7f0b301991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b35994700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 0x7f0b30199720 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0057c0 tx=0x7f0b2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c027070 con 0x7f0b301034b0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b2c004020 con 0x7f0b301034b0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b3019e480 con 0x7f0b301034b0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2c022c40 con 0x7f0b301034b0 2024-04-25T20:57:46.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.326+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b301a5d60 con 0x7f0b301034b0 2024-04-25T20:57:46.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.327+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b30069730 con 0x7f0b301034b0 2024-04-25T20:57:46.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.329+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f0b2c02a070 con 0x7f0b301034b0 2024-04-25T20:57:46.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.329+0000 7f0b277fe700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0b1c0a3ed0 0x7f0b1c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:46.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.330+0000 7f0b36195700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0b1c0a3ed0 0x7f0b1c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:46.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.330+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f0b2c0d2190 con 0x7f0b301034b0 2024-04-25T20:57:46.331 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.330+0000 7f0b36195700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0b1c0a3ed0 0x7f0b1c0a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005e10 tx=0x7f0b20005d60 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:46.333 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.333+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0b2c031020 con 0x7f0b301034b0 2024-04-25T20:57:46.531 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.530+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b3019a540 con 0x7f0b1c0a3ed0 2024-04-25T20:57:46.537 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.537+0000 7f0b277fe700 1 -- 172.21.15.96:0/4127533054 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257741 (secure 0 0 0) 0x7f0b3019a540 con 0x7f0b1c0a3ed0 2024-04-25T20:57:46.542 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0b1c0a3ed0 msgr2=0x7f0b1c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0b1c0a3ed0 0x7f0b1c0a6390 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b20005e10 tx=0x7f0b20005d60 comp rx=0 tx=0).stop 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 msgr2=0x7f0b30199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 0x7f0b30199720 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c0057c0 tx=0x7f0b2c0057f0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 shutdown_connections 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f0b1c0a3ed0 0x7f0b1c0a6390 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0b30108a80 0x7f0b3019dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0b30102af0 0x7f0b301991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 --2- 172.21.15.96:0/4127533054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0b301034b0 0x7f0b30199720 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 >> 172.21.15.96:0/4127533054 conn(0x7f0b30079b20 msgr2=0x7f0b301018d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:46.544 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 shutdown_connections 2024-04-25T20:57:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:46.540+0000 7f0b383f9700 1 -- 172.21.15.96:0/4127533054 wait complete. 2024-04-25T20:57:46.545 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:57:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:46 smithi096 ceph-mon[29288]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:46.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:46 smithi097 ceph-mon[32667]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:46 smithi139 ceph-mon[33027]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T20:57:47.236 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-04-25T20:57:44.536201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1187318,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1298,"num_read_kb":808,"num_write":476,"num_write_kb":4378,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":389928,"kb_used_data":15728,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551896,"statfs":{"total":1151940427776,"available":1151541141504,"internally_reserved":0,"allocated":16105472,"data_stored":9594478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.987708"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191422+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:57:39.191422+0000","last_peered":"2024-04-25T20:57:39.191422+0000","last_clean":"2024-04-25T20:57:39.191422+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:57:39.191422+0000","last_undegraded":"2024-04-25T20:57:39.191422+0000","last_fullsized":"2024-04-25T20:57:39.191422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691744+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:57:38.691744+0000","last_peered":"2024-04-25T20:57:38.691744+0000","last_clean":"2024-04-25T20:57:38.691744+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:57:38.691744+0000","last_undegraded":"2024-04-25T20:57:38.691744+0000","last_fullsized":"2024-04-25T20:57:38.691744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691501+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:57:38.691501+0000","last_peered":"2024-04-25T20:57:38.691501+0000","last_clean":"2024-04-25T20:57:38.691501+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:57:38.691501+0000","last_undegraded":"2024-04-25T20:57:38.691501+0000","last_fullsized":"2024-04-25T20:57:38.691501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616317+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:57:38.616317+0000","last_peered":"2024-04-25T20:57:38.616317+0000","last_clean":"2024-04-25T20:57:38.616317+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:57:38.616317+0000","last_undegraded":"2024-04-25T20:57:38.616317+0000","last_fullsized":"2024-04-25T20:57:38.616317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622554+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:57:38.622554+0000","last_peered":"2024-04-25T20:57:38.622554+0000","last_clean":"2024-04-25T20:57:38.622554+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:57:38.622554+0000","last_undegraded":"2024-04-25T20:57:38.622554+0000","last_fullsized":"2024-04-25T20:57:38.622554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522256+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:57:38.522256+0000","last_peered":"2024-04-25T20:57:38.522256+0000","last_clean":"2024-04-25T20:57:38.522256+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:57:38.522256+0000","last_undegraded":"2024-04-25T20:57:38.522256+0000","last_fullsized":"2024-04-25T20:57:38.522256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691132+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:57:38.691132+0000","last_peered":"2024-04-25T20:57:38.691132+0000","last_clean":"2024-04-25T20:57:38.691132+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:57:38.691132+0000","last_undegraded":"2024-04-25T20:57:38.691132+0000","last_fullsized":"2024-04-25T20:57:38.691132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523035+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:57:38.523035+0000","last_peered":"2024-04-25T20:57:38.523035+0000","last_clean":"2024-04-25T20:57:38.523035+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:57:38.523035+0000","last_undegraded":"2024-04-25T20:57:38.523035+0000","last_fullsized":"2024-04-25T20:57:38.523035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692124+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:57:38.692124+0000","last_peered":"2024-04-25T20:57:38.692124+0000","last_clean":"2024-04-25T20:57:38.692124+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:57:38.692124+0000","last_undegraded":"2024-04-25T20:57:38.692124+0000","last_fullsized":"2024-04-25T20:57:38.692124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522625+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:57:38.522625+0000","last_peered":"2024-04-25T20:57:38.522625+0000","last_clean":"2024-04-25T20:57:38.522625+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:57:38.522625+0000","last_undegraded":"2024-04-25T20:57:38.522625+0000","last_fullsized":"2024-04-25T20:57:38.522625+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692177+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:57:38.692177+0000","last_peered":"2024-04-25T20:57:38.692177+0000","last_clean":"2024-04-25T20:57:38.692177+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:57:38.692177+0000","last_undegraded":"2024-04-25T20:57:38.692177+0000","last_fullsized":"2024-04-25T20:57:38.692177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616238+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:57:38.616238+0000","last_peered":"2024-04-25T20:57:38.616238+0000","last_clean":"2024-04-25T20:57:38.616238+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:57:38.616238+0000","last_undegraded":"2024-04-25T20:57:38.616238+0000","last_fullsized":"2024-04-25T20:57:38.616238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614743+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:57:38.614743+0000","last_peered":"2024-04-25T20:57:38.614743+0000","last_clean":"2024-04-25T20:57:38.614743+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:57:38.614743+0000","last_undegraded":"2024-04-25T20:57:38.614743+0000","last_fullsized":"2024-04-25T20:57:38.614743+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691157+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:57:38.691157+0000","last_peered":"2024-04-25T20:57:38.691157+0000","last_clean":"2024-04-25T20:57:38.691157+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:57:38.691157+0000","last_undegraded":"2024-04-25T20:57:38.691157+0000","last_fullsized":"2024-04-25T20:57:38.691157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691059+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:57:38.691059+0000","last_peered":"2024-04-25T20:57:38.691059+0000","last_clean":"2024-04-25T20:57:38.691059+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:57:38.691059+0000","last_undegraded":"2024-04-25T20:57:38.691059+0000","last_fullsized":"2024-04-25T20:57:38.691059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691068+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:57:38.691068+0000","last_peered":"2024-04-25T20:57:38.691068+0000","last_clean":"2024-04-25T20:57:38.691068+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:57:38.691068+0000","last_undegraded":"2024-04-25T20:57:38.691068+0000","last_fullsized":"2024-04-25T20:57:38.691068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191562+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:57:39.191562+0000","last_peered":"2024-04-25T20:57:39.191562+0000","last_clean":"2024-04-25T20:57:39.191562+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:57:39.191562+0000","last_undegraded":"2024-04-25T20:57:39.191562+0000","last_fullsized":"2024-04-25T20:57:39.191562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.620534+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:57:38.620534+0000","last_peered":"2024-04-25T20:57:38.620534+0000","last_clean":"2024-04-25T20:57:38.620534+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:57:38.620534+0000","last_undegraded":"2024-04-25T20:57:38.620534+0000","last_fullsized":"2024-04-25T20:57:38.620534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524590+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:57:38.524590+0000","last_peered":"2024-04-25T20:57:38.524590+0000","last_clean":"2024-04-25T20:57:38.524590+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:57:38.524590+0000","last_undegraded":"2024-04-25T20:57:38.524590+0000","last_fullsized":"2024-04-25T20:57:38.524590+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.620316+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:57:38.620316+0000","last_peered":"2024-04-25T20:57:38.620316+0000","last_clean":"2024-04-25T20:57:38.620316+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:57:38.620316+0000","last_undegraded":"2024-04-25T20:57:38.620316+0000","last_fullsized":"2024-04-25T20:57:38.620316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523372+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:57:38.523372+0000","last_peered":"2024-04-25T20:57:38.523372+0000","last_clean":"2024-04-25T20:57:38.523372+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:57:38.523372+0000","last_undegraded":"2024-04-25T20:57:38.523372+0000","last_fullsized":"2024-04-25T20:57:38.523372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523534+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:57:38.523534+0000","last_peered":"2024-04-25T20:57:38.523534+0000","last_clean":"2024-04-25T20:57:38.523534+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:57:38.523534+0000","last_undegraded":"2024-04-25T20:57:38.523534+0000","last_fullsized":"2024-04-25T20:57:38.523534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691859+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:57:38.691859+0000","last_peered":"2024-04-25T20:57:38.691859+0000","last_clean":"2024-04-25T20:57:38.691859+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:57:38.691859+0000","last_undegraded":"2024-04-25T20:57:38.691859+0000","last_fullsized":"2024-04-25T20:57:38.691859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622881+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:57:38.622881+0000","last_peered":"2024-04-25T20:57:38.622881+0000","last_clean":"2024-04-25T20:57:38.622881+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:57:38.622881+0000","last_undegraded":"2024-04-25T20:57:38.622881+0000","last_fullsized":"2024-04-25T20:57:38.622881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691786+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:57:38.691786+0000","last_peered":"2024-04-25T20:57:38.691786+0000","last_clean":"2024-04-25T20:57:38.691786+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:57:38.691786+0000","last_undegraded":"2024-04-25T20:57:38.691786+0000","last_fullsized":"2024-04-25T20:57:38.691786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622783+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:57:38.622783+0000","last_peered":"2024-04-25T20:57:38.622783+0000","last_clean":"2024-04-25T20:57:38.622783+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:57:38.622783+0000","last_undegraded":"2024-04-25T20:57:38.622783+0000","last_fullsized":"2024-04-25T20:57:38.622783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616254+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:57:38.616254+0000","last_peered":"2024-04-25T20:57:38.616254+0000","last_clean":"2024-04-25T20:57:38.616254+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:57:38.616254+0000","last_undegraded":"2024-04-25T20:57:38.616254+0000","last_fullsized":"2024-04-25T20:57:38.616254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":216,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616381+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:57:38.616381+0000","last_peered":"2024-04-25T20:57:38.616381+0000","last_clean":"2024-04-25T20:57:38.616381+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:57:38.616381+0000","last_undegraded":"2024-04-25T20:57:38.616381+0000","last_fullsized":"2024-04-25T20:57:38.616381+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1853,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622512+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:57:38.622512+0000","last_peered":"2024-04-25T20:57:38.622512+0000","last_clean":"2024-04-25T20:57:38.622512+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:57:38.622512+0000","last_undegraded":"2024-04-25T20:57:38.622512+0000","last_fullsized":"2024-04-25T20:57:38.622512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522207+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:57:38.522207+0000","last_peered":"2024-04-25T20:57:38.522207+0000","last_clean":"2024-04-25T20:57:38.522207+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:57:38.522207+0000","last_undegraded":"2024-04-25T20:57:38.522207+0000","last_fullsized":"2024-04-25T20:57:38.522207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615952+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:57:38.615952+0000","last_peered":"2024-04-25T20:57:38.615952+0000","last_clean":"2024-04-25T20:57:38.615952+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:57:38.615952+0000","last_undegraded":"2024-04-25T20:57:38.615952+0000","last_fullsized":"2024-04-25T20:57:38.615952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614503+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:57:38.614503+0000","last_peered":"2024-04-25T20:57:38.614503+0000","last_clean":"2024-04-25T20:57:38.614503+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:57:38.614503+0000","last_undegraded":"2024-04-25T20:57:38.614503+0000","last_fullsized":"2024-04-25T20:57:38.614503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522962+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:57:38.522962+0000","last_peered":"2024-04-25T20:57:38.522962+0000","last_clean":"2024-04-25T20:57:38.522962+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:57:38.522962+0000","last_undegraded":"2024-04-25T20:57:38.522962+0000","last_fullsized":"2024-04-25T20:57:38.522962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615783+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:57:38.615783+0000","last_peered":"2024-04-25T20:57:38.615783+0000","last_clean":"2024-04-25T20:57:38.615783+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:57:38.615783+0000","last_undegraded":"2024-04-25T20:57:38.615783+0000","last_fullsized":"2024-04-25T20:57:38.615783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.619924+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:57:38.619924+0000","last_peered":"2024-04-25T20:57:38.619924+0000","last_clean":"2024-04-25T20:57:38.619924+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:57:38.619924+0000","last_undegraded":"2024-04-25T20:57:38.619924+0000","last_fullsized":"2024-04-25T20:57:38.619924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":206,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522702+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:57:38.522702+0000","last_peered":"2024-04-25T20:57:38.522702+0000","last_clean":"2024-04-25T20:57:38.522702+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:57:38.522702+0000","last_undegraded":"2024-04-25T20:57:38.522702+0000","last_fullsized":"2024-04-25T20:57:38.522702+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691858+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:57:38.691858+0000","last_peered":"2024-04-25T20:57:38.691858+0000","last_clean":"2024-04-25T20:57:38.691858+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:57:38.691858+0000","last_undegraded":"2024-04-25T20:57:38.691858+0000","last_fullsized":"2024-04-25T20:57:38.691858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692223+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:57:38.692223+0000","last_peered":"2024-04-25T20:57:38.692223+0000","last_clean":"2024-04-25T20:57:38.692223+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:57:38.692223+0000","last_undegraded":"2024-04-25T20:57:38.692223+0000","last_fullsized":"2024-04-25T20:57:38.692223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691875+0000","last_change":"2024-04-25T20:55:10.334414+0000","last_active":"2024-04-25T20:57:38.691875+0000","last_peered":"2024-04-25T20:57:38.691875+0000","last_clean":"2024-04-25T20:57:38.691875+0000","last_became_active":"2024-04-25T20:55:10.334022+0000","last_became_peered":"2024-04-25T20:55:10.334022+0000","last_unstale":"2024-04-25T20:57:38.691875+0000","last_undegraded":"2024-04-25T20:57:38.691875+0000","last_fullsized":"2024-04-25T20:57:38.691875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:32.914876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615739+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:57:38.615739+0000","last_peered":"2024-04-25T20:57:38.615739+0000","last_clean":"2024-04-25T20:57:38.615739+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:57:38.615739+0000","last_undegraded":"2024-04-25T20:57:38.615739+0000","last_fullsized":"2024-04-25T20:57:38.615739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691880+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:57:38.691880+0000","last_peered":"2024-04-25T20:57:38.691880+0000","last_clean":"2024-04-25T20:57:38.691880+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:57:38.691880+0000","last_undegraded":"2024-04-25T20:57:38.691880+0000","last_fullsized":"2024-04-25T20:57:38.691880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'37","reported_seq":423,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.769802+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:57:38.769802+0000","last_peered":"2024-04-25T20:57:38.769802+0000","last_clean":"2024-04-25T20:57:38.769802+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:57:38.769802+0000","last_undegraded":"2024-04-25T20:57:38.769802+0000","last_fullsized":"2024-04-25T20:57:38.769802+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":358,"num_read_kb":244,"num_write":38,"num_write_kb":83,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622687+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:57:38.622687+0000","last_peered":"2024-04-25T20:57:38.622687+0000","last_clean":"2024-04-25T20:57:38.622687+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:57:38.622687+0000","last_undegraded":"2024-04-25T20:57:38.622687+0000","last_fullsized":"2024-04-25T20:57:38.622687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524419+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:57:38.524419+0000","last_peered":"2024-04-25T20:57:38.524419+0000","last_clean":"2024-04-25T20:57:38.524419+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:57:38.524419+0000","last_undegraded":"2024-04-25T20:57:38.524419+0000","last_fullsized":"2024-04-25T20:57:38.524419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614481+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:57:38.614481+0000","last_peered":"2024-04-25T20:57:38.614481+0000","last_clean":"2024-04-25T20:57:38.614481+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:57:38.614481+0000","last_undegraded":"2024-04-25T20:57:38.614481+0000","last_fullsized":"2024-04-25T20:57:38.614481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616027+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:57:38.616027+0000","last_peered":"2024-04-25T20:57:38.616027+0000","last_clean":"2024-04-25T20:57:38.616027+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:57:38.616027+0000","last_undegraded":"2024-04-25T20:57:38.616027+0000","last_fullsized":"2024-04-25T20:57:38.616027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691679+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:57:38.691679+0000","last_peered":"2024-04-25T20:57:38.691679+0000","last_clean":"2024-04-25T20:57:38.691679+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:57:38.691679+0000","last_undegraded":"2024-04-25T20:57:38.691679+0000","last_fullsized":"2024-04-25T20:57:38.691679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"91'3","reported_seq":206,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616237+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:57:38.616237+0000","last_peered":"2024-04-25T20:57:38.616237+0000","last_clean":"2024-04-25T20:57:38.616237+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:57:38.616237+0000","last_undegraded":"2024-04-25T20:57:38.616237+0000","last_fullsized":"2024-04-25T20:57:38.616237+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":176,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691075+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:57:38.691075+0000","last_peered":"2024-04-25T20:57:38.691075+0000","last_clean":"2024-04-25T20:57:38.691075+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:57:38.691075+0000","last_undegraded":"2024-04-25T20:57:38.691075+0000","last_fullsized":"2024-04-25T20:57:38.691075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622432+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:57:38.622432+0000","last_peered":"2024-04-25T20:57:38.622432+0000","last_clean":"2024-04-25T20:57:38.622432+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:57:38.622432+0000","last_undegraded":"2024-04-25T20:57:38.622432+0000","last_fullsized":"2024-04-25T20:57:38.622432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615853+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:57:38.615853+0000","last_peered":"2024-04-25T20:57:38.615853+0000","last_clean":"2024-04-25T20:57:38.615853+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:57:38.615853+0000","last_undegraded":"2024-04-25T20:57:38.615853+0000","last_fullsized":"2024-04-25T20:57:38.615853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622920+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:57:38.622920+0000","last_peered":"2024-04-25T20:57:38.622920+0000","last_clean":"2024-04-25T20:57:38.622920+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:57:38.622920+0000","last_undegraded":"2024-04-25T20:57:38.622920+0000","last_fullsized":"2024-04-25T20:57:38.622920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191264+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:57:39.191264+0000","last_peered":"2024-04-25T20:57:39.191264+0000","last_clean":"2024-04-25T20:57:39.191264+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:57:39.191264+0000","last_undegraded":"2024-04-25T20:57:39.191264+0000","last_fullsized":"2024-04-25T20:57:39.191264+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.619738+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:57:38.619738+0000","last_peered":"2024-04-25T20:57:38.619738+0000","last_clean":"2024-04-25T20:57:38.619738+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:57:38.619738+0000","last_undegraded":"2024-04-25T20:57:38.619738+0000","last_fullsized":"2024-04-25T20:57:38.619738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522451+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:57:38.522451+0000","last_peered":"2024-04-25T20:57:38.522451+0000","last_clean":"2024-04-25T20:57:38.522451+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:57:38.522451+0000","last_undegraded":"2024-04-25T20:57:38.522451+0000","last_fullsized":"2024-04-25T20:57:38.522451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691667+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:57:38.691667+0000","last_peered":"2024-04-25T20:57:38.691667+0000","last_clean":"2024-04-25T20:57:38.691667+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:57:38.691667+0000","last_undegraded":"2024-04-25T20:57:38.691667+0000","last_fullsized":"2024-04-25T20:57:38.691667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522266+0000","last_change":"2024-04-25T20:55:10.332432+0000","last_active":"2024-04-25T20:57:38.522266+0000","last_peered":"2024-04-25T20:57:38.522266+0000","last_clean":"2024-04-25T20:57:38.522266+0000","last_became_active":"2024-04-25T20:55:10.332033+0000","last_became_peered":"2024-04-25T20:55:10.332033+0000","last_unstale":"2024-04-25T20:57:38.522266+0000","last_undegraded":"2024-04-25T20:57:38.522266+0000","last_fullsized":"2024-04-25T20:57:38.522266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:34.930705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616167+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:57:38.616167+0000","last_peered":"2024-04-25T20:57:38.616167+0000","last_clean":"2024-04-25T20:57:38.616167+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:57:38.616167+0000","last_undegraded":"2024-04-25T20:57:38.616167+0000","last_fullsized":"2024-04-25T20:57:38.616167+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'4","reported_seq":206,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614347+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:57:38.614347+0000","last_peered":"2024-04-25T20:57:38.614347+0000","last_clean":"2024-04-25T20:57:38.614347+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:57:38.614347+0000","last_undegraded":"2024-04-25T20:57:38.614347+0000","last_fullsized":"2024-04-25T20:57:38.614347+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":173,"num_read_kb":173,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523216+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:57:38.523216+0000","last_peered":"2024-04-25T20:57:38.523216+0000","last_clean":"2024-04-25T20:57:38.523216+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:57:38.523216+0000","last_undegraded":"2024-04-25T20:57:38.523216+0000","last_fullsized":"2024-04-25T20:57:38.523216+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616224+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:57:38.616224+0000","last_peered":"2024-04-25T20:57:38.616224+0000","last_clean":"2024-04-25T20:57:38.616224+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:57:38.616224+0000","last_undegraded":"2024-04-25T20:57:38.616224+0000","last_fullsized":"2024-04-25T20:57:38.616224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":364,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691261+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:57:38.691261+0000","last_peered":"2024-04-25T20:57:38.691261+0000","last_clean":"2024-04-25T20:57:38.691261+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:57:38.691261+0000","last_undegraded":"2024-04-25T20:57:38.691261+0000","last_fullsized":"2024-04-25T20:57:38.691261+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614336+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:57:38.614336+0000","last_peered":"2024-04-25T20:57:38.614336+0000","last_clean":"2024-04-25T20:57:38.614336+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:57:38.614336+0000","last_undegraded":"2024-04-25T20:57:38.614336+0000","last_fullsized":"2024-04-25T20:57:38.614336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691380+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:57:38.691380+0000","last_peered":"2024-04-25T20:57:38.691380+0000","last_clean":"2024-04-25T20:57:38.691380+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:57:38.691380+0000","last_undegraded":"2024-04-25T20:57:38.691380+0000","last_fullsized":"2024-04-25T20:57:38.691380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191351+0000","last_change":"2024-04-25T20:55:10.337450+0000","last_active":"2024-04-25T20:57:39.191351+0000","last_peered":"2024-04-25T20:57:39.191351+0000","last_clean":"2024-04-25T20:57:39.191351+0000","last_became_active":"2024-04-25T20:55:10.332821+0000","last_became_peered":"2024-04-25T20:55:10.332821+0000","last_unstale":"2024-04-25T20:57:39.191351+0000","last_undegraded":"2024-04-25T20:57:39.191351+0000","last_fullsized":"2024-04-25T20:57:39.191351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:55.070702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614931+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:57:38.614931+0000","last_peered":"2024-04-25T20:57:38.614931+0000","last_clean":"2024-04-25T20:57:38.614931+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:57:38.614931+0000","last_undegraded":"2024-04-25T20:57:38.614931+0000","last_fullsized":"2024-04-25T20:57:38.614931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":100,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.739095+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:57:38.739095+0000","last_peered":"2024-04-25T20:57:38.739095+0000","last_clean":"2024-04-25T20:57:38.739095+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:57:38.739095+0000","last_undegraded":"2024-04-25T20:57:38.739095+0000","last_fullsized":"2024-04-25T20:57:38.739095+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":87,"num_write_kb":839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523457+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:57:38.523457+0000","last_peered":"2024-04-25T20:57:38.523457+0000","last_clean":"2024-04-25T20:57:38.523457+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:57:38.523457+0000","last_undegraded":"2024-04-25T20:57:38.523457+0000","last_fullsized":"2024-04-25T20:57:38.523457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616365+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:57:38.616365+0000","last_peered":"2024-04-25T20:57:38.616365+0000","last_clean":"2024-04-25T20:57:38.616365+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:57:38.616365+0000","last_undegraded":"2024-04-25T20:57:38.616365+0000","last_fullsized":"2024-04-25T20:57:38.616365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691756+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:57:38.691756+0000","last_peered":"2024-04-25T20:57:38.691756+0000","last_clean":"2024-04-25T20:57:38.691756+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:57:38.691756+0000","last_undegraded":"2024-04-25T20:57:38.691756+0000","last_fullsized":"2024-04-25T20:57:38.691756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691035+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:57:38.691035+0000","last_peered":"2024-04-25T20:57:38.691035+0000","last_clean":"2024-04-25T20:57:38.691035+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:57:38.691035+0000","last_undegraded":"2024-04-25T20:57:38.691035+0000","last_fullsized":"2024-04-25T20:57:38.691035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622483+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:57:38.622483+0000","last_peered":"2024-04-25T20:57:38.622483+0000","last_clean":"2024-04-25T20:57:38.622483+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:57:38.622483+0000","last_undegraded":"2024-04-25T20:57:38.622483+0000","last_fullsized":"2024-04-25T20:57:38.622483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523034+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:57:38.523034+0000","last_peered":"2024-04-25T20:57:38.523034+0000","last_clean":"2024-04-25T20:57:38.523034+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:57:38.523034+0000","last_undegraded":"2024-04-25T20:57:38.523034+0000","last_fullsized":"2024-04-25T20:57:38.523034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191000+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:57:39.191000+0000","last_peered":"2024-04-25T20:57:39.191000+0000","last_clean":"2024-04-25T20:57:39.191000+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:57:39.191000+0000","last_undegraded":"2024-04-25T20:57:39.191000+0000","last_fullsized":"2024-04-25T20:57:39.191000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'2","reported_seq":19,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191125+0000","last_change":"2024-04-25T20:55:10.339729+0000","last_active":"2024-04-25T20:57:39.191125+0000","last_peered":"2024-04-25T20:57:39.191125+0000","last_clean":"2024-04-25T20:57:39.191125+0000","last_became_active":"2024-04-25T20:55:10.333756+0000","last_became_peered":"2024-04-25T20:55:10.333756+0000","last_unstale":"2024-04-25T20:57:39.191125+0000","last_undegraded":"2024-04-25T20:57:39.191125+0000","last_fullsized":"2024-04-25T20:57:39.191125+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:48.806227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523332+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:57:38.523332+0000","last_peered":"2024-04-25T20:57:38.523332+0000","last_clean":"2024-04-25T20:57:38.523332+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:57:38.523332+0000","last_undegraded":"2024-04-25T20:57:38.523332+0000","last_fullsized":"2024-04-25T20:57:38.523332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622600+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:57:38.622600+0000","last_peered":"2024-04-25T20:57:38.622600+0000","last_clean":"2024-04-25T20:57:38.622600+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:57:38.622600+0000","last_undegraded":"2024-04-25T20:57:38.622600+0000","last_fullsized":"2024-04-25T20:57:38.622600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691778+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:57:38.691778+0000","last_peered":"2024-04-25T20:57:38.691778+0000","last_clean":"2024-04-25T20:57:38.691778+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:57:38.691778+0000","last_undegraded":"2024-04-25T20:57:38.691778+0000","last_fullsized":"2024-04-25T20:57:38.691778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524689+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:57:38.524689+0000","last_peered":"2024-04-25T20:57:38.524689+0000","last_clean":"2024-04-25T20:57:38.524689+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:57:38.524689+0000","last_undegraded":"2024-04-25T20:57:38.524689+0000","last_fullsized":"2024-04-25T20:57:38.524689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692261+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:57:38.692261+0000","last_peered":"2024-04-25T20:57:38.692261+0000","last_clean":"2024-04-25T20:57:38.692261+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:57:38.692261+0000","last_undegraded":"2024-04-25T20:57:38.692261+0000","last_fullsized":"2024-04-25T20:57:38.692261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'39","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.732731+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:57:38.732731+0000","last_peered":"2024-04-25T20:57:38.732731+0000","last_clean":"2024-04-25T20:57:38.732731+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:57:38.732731+0000","last_undegraded":"2024-04-25T20:57:38.732731+0000","last_fullsized":"2024-04-25T20:57:38.732731+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691515+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:57:38.691515+0000","last_peered":"2024-04-25T20:57:38.691515+0000","last_clean":"2024-04-25T20:57:38.691515+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:57:38.691515+0000","last_undegraded":"2024-04-25T20:57:38.691515+0000","last_fullsized":"2024-04-25T20:57:38.691515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614981+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:57:38.614981+0000","last_peered":"2024-04-25T20:57:38.614981+0000","last_clean":"2024-04-25T20:57:38.614981+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:57:38.614981+0000","last_undegraded":"2024-04-25T20:57:38.614981+0000","last_fullsized":"2024-04-25T20:57:38.614981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692338+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:57:38.692338+0000","last_peered":"2024-04-25T20:57:38.692338+0000","last_clean":"2024-04-25T20:57:38.692338+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:57:38.692338+0000","last_undegraded":"2024-04-25T20:57:38.692338+0000","last_fullsized":"2024-04-25T20:57:38.692338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191053+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:57:39.191053+0000","last_peered":"2024-04-25T20:57:39.191053+0000","last_clean":"2024-04-25T20:57:39.191053+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:57:39.191053+0000","last_undegraded":"2024-04-25T20:57:39.191053+0000","last_fullsized":"2024-04-25T20:57:39.191053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524295+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:57:38.524295+0000","last_peered":"2024-04-25T20:57:38.524295+0000","last_clean":"2024-04-25T20:57:38.524295+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:57:38.524295+0000","last_undegraded":"2024-04-25T20:57:38.524295+0000","last_fullsized":"2024-04-25T20:57:38.524295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524391+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:57:38.524391+0000","last_peered":"2024-04-25T20:57:38.524391+0000","last_clean":"2024-04-25T20:57:38.524391+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:57:38.524391+0000","last_undegraded":"2024-04-25T20:57:38.524391+0000","last_fullsized":"2024-04-25T20:57:38.524391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691369+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:57:38.691369+0000","last_peered":"2024-04-25T20:57:38.691369+0000","last_clean":"2024-04-25T20:57:38.691369+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:57:38.691369+0000","last_undegraded":"2024-04-25T20:57:38.691369+0000","last_fullsized":"2024-04-25T20:57:38.691369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614940+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:57:38.614940+0000","last_peered":"2024-04-25T20:57:38.614940+0000","last_clean":"2024-04-25T20:57:38.614940+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:57:38.614940+0000","last_undegraded":"2024-04-25T20:57:38.614940+0000","last_fullsized":"2024-04-25T20:57:38.614940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191270+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:57:39.191270+0000","last_peered":"2024-04-25T20:57:39.191270+0000","last_clean":"2024-04-25T20:57:39.191270+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:57:39.191270+0000","last_undegraded":"2024-04-25T20:57:39.191270+0000","last_fullsized":"2024-04-25T20:57:39.191270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524541+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:57:38.524541+0000","last_peered":"2024-04-25T20:57:38.524541+0000","last_clean":"2024-04-25T20:57:38.524541+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:57:38.524541+0000","last_undegraded":"2024-04-25T20:57:38.524541+0000","last_fullsized":"2024-04-25T20:57:38.524541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523416+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:57:38.523416+0000","last_peered":"2024-04-25T20:57:38.523416+0000","last_clean":"2024-04-25T20:57:38.523416+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:57:38.523416+0000","last_undegraded":"2024-04-25T20:57:38.523416+0000","last_fullsized":"2024-04-25T20:57:38.523416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615843+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:57:38.615843+0000","last_peered":"2024-04-25T20:57:38.615843+0000","last_clean":"2024-04-25T20:57:38.615843+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:57:38.615843+0000","last_undegraded":"2024-04-25T20:57:38.615843+0000","last_fullsized":"2024-04-25T20:57:38.615843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522987+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:57:38.522987+0000","last_peered":"2024-04-25T20:57:38.522987+0000","last_clean":"2024-04-25T20:57:38.522987+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:57:38.522987+0000","last_undegraded":"2024-04-25T20:57:38.522987+0000","last_fullsized":"2024-04-25T20:57:38.522987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691387+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:57:38.691387+0000","last_peered":"2024-04-25T20:57:38.691387+0000","last_clean":"2024-04-25T20:57:38.691387+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:57:38.691387+0000","last_undegraded":"2024-04-25T20:57:38.691387+0000","last_fullsized":"2024-04-25T20:57:38.691387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614712+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:57:38.614712+0000","last_peered":"2024-04-25T20:57:38.614712+0000","last_clean":"2024-04-25T20:57:38.614712+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:57:38.614712+0000","last_undegraded":"2024-04-25T20:57:38.614712+0000","last_fullsized":"2024-04-25T20:57:38.614712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191048+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:57:39.191048+0000","last_peered":"2024-04-25T20:57:39.191048+0000","last_clean":"2024-04-25T20:57:39.191048+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:57:39.191048+0000","last_undegraded":"2024-04-25T20:57:39.191048+0000","last_fullsized":"2024-04-25T20:57:39.191048+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596716,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1162,"num_read_kb":689,"num_write":293,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2027520,"data_stored":1790148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285623,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":495217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547260,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37160,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707992,"statfs":{"total":95995035648,"available":95956983808,"internally_reserved":0,"allocated":1867776,"data_stored":1294204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743488,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36744,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708408,"statfs":{"total":95995035648,"available":95957409792,"internally_reserved":0,"allocated":1437696,"data_stored":897516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005124,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32576,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712576,"statfs":{"total":95995035648,"available":95961677824,"internally_reserved":0,"allocated":1433600,"data_stored":897414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1096599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462992,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":503689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36416,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708736,"statfs":{"total":95995035648,"available":95957745664,"internally_reserved":0,"allocated":1101824,"data_stored":570541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738478,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":504960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411428,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":573122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149790,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28556,"kb_used_data":1540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716596,"statfs":{"total":95995035648,"available":95965794304,"internally_reserved":0,"allocated":1576960,"data_stored":1028816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888153,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":504126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724629,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28820,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716332,"statfs":{"total":95995035648,"available":95965523968,"internally_reserved":0,"allocated":1785856,"data_stored":1228274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":544768,"data_stored":525100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":163840,"data_stored":134307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":245760,"data_stored":200237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:57:47.239 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-25T20:57:47.240 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-25T20:57:47.483 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:47.831 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:47 smithi096 ceph-mon[29288]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:47.831 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:47 smithi096 ceph-mon[29288]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T20:57:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:47 smithi097 ceph-mon[32667]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:47 smithi097 ceph-mon[32667]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T20:57:47.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:47 smithi139 ceph-mon[33027]: from='client.15261 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:47.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:47 smithi139 ceph-mon[33027]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 77 op/s 2024-04-25T20:57:49.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/430439990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0107990 msgr2=0x7f7ef0107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/430439990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0107990 0x7f7ef0107d70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f7eec009990 tx=0x7f7eec009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.132 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/430439990 shutdown_connections 2024-04-25T20:57:49.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/430439990 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0102300 0x7f7ef010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/430439990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0107990 0x7f7ef0107d70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/430439990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef0101940 0x7f7ef0101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/430439990 >> 172.21.15.96:0/430439990 conn(0x7f7ef00788f0 msgr2=0x7f7ef0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:49.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/430439990 shutdown_connections 2024-04-25T20:57:49.133 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.131+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/430439990 wait complete. 2024-04-25T20:57:49.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 Processor -- start 2024-04-25T20:57:49.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 -- start start 2024-04-25T20:57:49.134 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 0x7f7ef0198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef0102300 0x7f7ef01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0107990 0x7f7ef019c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0111b70 con 0x7f7ef0101940 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef01119f0 con 0x7f7ef0102300 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef7ebb700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0111cf0 con 0x7f7ef0107990 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef5c57700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 0x7f7ef0198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef5c57700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 0x7f7ef0198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33184/0 (socket says 172.21.15.96:33184) 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef5c57700 1 -- 172.21.15.96:0/3331928835 learned_addr learned my addr 172.21.15.96:0/3331928835 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef5456700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef0102300 0x7f7ef01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.132+0000 7f7ef6458700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0107990 0x7f7ef019c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef5c57700 1 -- 172.21.15.96:0/3331928835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0107990 msgr2=0x7f7ef019c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef5c57700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0107990 0x7f7ef019c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef5c57700 1 -- 172.21.15.96:0/3331928835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef0102300 msgr2=0x7f7ef01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.135 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef5c57700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef0102300 0x7f7ef01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef5c57700 1 -- 172.21.15.96:0/3331928835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef019d090 con 0x7f7ef0101940 2024-04-25T20:57:49.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef6458700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0107990 0x7f7ef019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:57:49.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef5c57700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 0x7f7ef0198080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000c00 tx=0x7f7eec0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:49.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec027070 con 0x7f7ef0101940 2024-04-25T20:57:49.136 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eec003f20 con 0x7f7ef0101940 2024-04-25T20:57:49.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef019d320 con 0x7f7ef0101940 2024-04-25T20:57:49.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec022cf0 con 0x7f7ef0101940 2024-04-25T20:57:49.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.133+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01a4c00 con 0x7f7ef0101940 2024-04-25T20:57:49.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.134+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef0109eb0 con 0x7f7ef0101940 2024-04-25T20:57:49.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.135+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f7eec02a070 con 0x7f7ef0101940 2024-04-25T20:57:49.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.136+0000 7f7ee6ffd700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f7edc0a3f50 0x7f7edc0a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:49.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.136+0000 7f7ef5456700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f7edc0a3f50 0x7f7edc0a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:49.138 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.137+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f7eec0d1b40 con 0x7f7ef0101940 2024-04-25T20:57:49.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.137+0000 7f7ef5456700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f7edc0a3f50 0x7f7edc0a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0078fd0 tx=0x7f7ee0009190 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:49.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.140+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7eec013070 con 0x7f7ef0101940 2024-04-25T20:57:49.334 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.333+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ef01993e0 con 0x7f7edc0a3f50 2024-04-25T20:57:49.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.341+0000 7f7ee6ffd700 1 -- 172.21.15.96:0/3331928835 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+257769 (secure 0 0 0) 0x7f7ef01993e0 con 0x7f7edc0a3f50 2024-04-25T20:57:49.342 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.343+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f7edc0a3f50 msgr2=0x7f7edc0a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.343+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f7edc0a3f50 0x7f7edc0a6410 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ef0078fd0 tx=0x7f7ee0009190 comp rx=0 tx=0).stop 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.343+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 msgr2=0x7f7ef0198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.343+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 0x7f7ef0198080 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f7eec000c00 tx=0x7f7eec0047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 shutdown_connections 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f7edc0a3f50 0x7f7edc0a6410 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f7ef0107990 0x7f7ef019c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ef0101940 0x7f7ef0198080 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 --2- 172.21.15.96:0/3331928835 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ef0102300 0x7f7ef01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 >> 172.21.15.96:0/3331928835 conn(0x7f7ef00788f0 msgr2=0x7f7ef0100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:49.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 shutdown_connections 2024-04-25T20:57:49.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:49.344+0000 7f7ef7ebb700 1 -- 172.21.15.96:0/3331928835 wait complete. 2024-04-25T20:57:49.348 INFO:teuthology.orchestra.run.smithi096.stderr:dumped all 2024-04-25T20:57:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:49 smithi097 ceph-mon[32667]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-25T20:57:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:49 smithi097 ceph-mon[32667]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:49 smithi139 ceph-mon[33027]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-25T20:57:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:49 smithi139 ceph-mon[33027]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:49.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:49 smithi096 ceph-mon[29288]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 103 KiB/s wr, 60 op/s 2024-04-25T20:57:49.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:49 smithi096 ceph-mon[29288]: from='client.15267 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-25T20:57:50.053 INFO:teuthology.orchestra.run.smithi096.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-25T20:57:48.538416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1387643,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1686,"num_read_kb":1271,"num_write":631,"num_write_kb":5301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":964,"ondisk_log_size":964,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":394972,"kb_used_data":16320,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124546852,"statfs":{"total":1151940427776,"available":1151535976448,"internally_reserved":0,"allocated":16711680,"data_stored":10204713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":387233084},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200325,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":388,"num_read_kb":463,"num_write":155,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.989922"},"pg_stats":[{"pgid":"3.3f","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191422+0000","last_change":"2024-04-25T20:54:29.394336+0000","last_active":"2024-04-25T20:57:39.191422+0000","last_peered":"2024-04-25T20:57:39.191422+0000","last_clean":"2024-04-25T20:57:39.191422+0000","last_became_active":"2024-04-25T20:54:29.393811+0000","last_became_peered":"2024-04-25T20:54:29.393811+0000","last_unstale":"2024-04-25T20:57:39.191422+0000","last_undegraded":"2024-04-25T20:57:39.191422+0000","last_fullsized":"2024-04-25T20:57:39.191422+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:30.225690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"93'4","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691744+0000","last_change":"2024-04-25T20:54:29.392555+0000","last_active":"2024-04-25T20:57:38.691744+0000","last_peered":"2024-04-25T20:57:38.691744+0000","last_clean":"2024-04-25T20:57:38.691744+0000","last_became_active":"2024-04-25T20:54:29.392244+0000","last_became_peered":"2024-04-25T20:54:29.392244+0000","last_unstale":"2024-04-25T20:57:38.691744+0000","last_undegraded":"2024-04-25T20:57:38.691744+0000","last_fullsized":"2024-04-25T20:57:38.691744+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:44:17.548479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691501+0000","last_change":"2024-04-25T20:54:29.401402+0000","last_active":"2024-04-25T20:57:38.691501+0000","last_peered":"2024-04-25T20:57:38.691501+0000","last_clean":"2024-04-25T20:57:38.691501+0000","last_became_active":"2024-04-25T20:54:29.401072+0000","last_became_peered":"2024-04-25T20:54:29.401072+0000","last_unstale":"2024-04-25T20:57:38.691501+0000","last_undegraded":"2024-04-25T20:57:38.691501+0000","last_fullsized":"2024-04-25T20:57:38.691501+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:17:29.341306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616317+0000","last_change":"2024-04-25T20:54:29.402745+0000","last_active":"2024-04-25T20:57:38.616317+0000","last_peered":"2024-04-25T20:57:38.616317+0000","last_clean":"2024-04-25T20:57:38.616317+0000","last_became_active":"2024-04-25T20:54:29.402212+0000","last_became_peered":"2024-04-25T20:54:29.402212+0000","last_unstale":"2024-04-25T20:57:38.616317+0000","last_undegraded":"2024-04-25T20:57:38.616317+0000","last_fullsized":"2024-04-25T20:57:38.616317+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:51:15.111584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"93'4","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622554+0000","last_change":"2024-04-25T20:54:29.388579+0000","last_active":"2024-04-25T20:57:38.622554+0000","last_peered":"2024-04-25T20:57:38.622554+0000","last_clean":"2024-04-25T20:57:38.622554+0000","last_became_active":"2024-04-25T20:54:29.388210+0000","last_became_peered":"2024-04-25T20:54:29.388210+0000","last_unstale":"2024-04-25T20:57:38.622554+0000","last_undegraded":"2024-04-25T20:57:38.622554+0000","last_fullsized":"2024-04-25T20:57:38.622554+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:37:54.240627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522256+0000","last_change":"2024-04-25T20:54:29.398488+0000","last_active":"2024-04-25T20:57:38.522256+0000","last_peered":"2024-04-25T20:57:38.522256+0000","last_clean":"2024-04-25T20:57:38.522256+0000","last_became_active":"2024-04-25T20:54:29.398160+0000","last_became_peered":"2024-04-25T20:54:29.398160+0000","last_unstale":"2024-04-25T20:57:38.522256+0000","last_undegraded":"2024-04-25T20:57:38.522256+0000","last_fullsized":"2024-04-25T20:57:38.522256+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:33:56.818266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691132+0000","last_change":"2024-04-25T20:54:29.402115+0000","last_active":"2024-04-25T20:57:38.691132+0000","last_peered":"2024-04-25T20:57:38.691132+0000","last_clean":"2024-04-25T20:57:38.691132+0000","last_became_active":"2024-04-25T20:54:29.401576+0000","last_became_peered":"2024-04-25T20:54:29.401576+0000","last_unstale":"2024-04-25T20:57:38.691132+0000","last_undegraded":"2024-04-25T20:57:38.691132+0000","last_fullsized":"2024-04-25T20:57:38.691132+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:46:19.278264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523035+0000","last_change":"2024-04-25T20:54:29.396149+0000","last_active":"2024-04-25T20:57:38.523035+0000","last_peered":"2024-04-25T20:57:38.523035+0000","last_clean":"2024-04-25T20:57:38.523035+0000","last_became_active":"2024-04-25T20:54:29.395816+0000","last_became_peered":"2024-04-25T20:54:29.395816+0000","last_unstale":"2024-04-25T20:57:38.523035+0000","last_undegraded":"2024-04-25T20:57:38.523035+0000","last_fullsized":"2024-04-25T20:57:38.523035+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:21:06.726991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692124+0000","last_change":"2024-04-25T20:54:29.402672+0000","last_active":"2024-04-25T20:57:38.692124+0000","last_peered":"2024-04-25T20:57:38.692124+0000","last_clean":"2024-04-25T20:57:38.692124+0000","last_became_active":"2024-04-25T20:54:29.402205+0000","last_became_peered":"2024-04-25T20:54:29.402205+0000","last_unstale":"2024-04-25T20:57:38.692124+0000","last_undegraded":"2024-04-25T20:57:38.692124+0000","last_fullsized":"2024-04-25T20:57:38.692124+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:39:55.969324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"93'16","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522625+0000","last_change":"2024-04-25T20:54:29.399665+0000","last_active":"2024-04-25T20:57:38.522625+0000","last_peered":"2024-04-25T20:57:38.522625+0000","last_clean":"2024-04-25T20:57:38.522625+0000","last_became_active":"2024-04-25T20:54:29.399265+0000","last_became_peered":"2024-04-25T20:54:29.399265+0000","last_unstale":"2024-04-25T20:57:38.522625+0000","last_undegraded":"2024-04-25T20:57:38.522625+0000","last_fullsized":"2024-04-25T20:57:38.522625+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:41:31.737709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692177+0000","last_change":"2024-04-25T20:54:29.391886+0000","last_active":"2024-04-25T20:57:38.692177+0000","last_peered":"2024-04-25T20:57:38.692177+0000","last_clean":"2024-04-25T20:57:38.692177+0000","last_became_active":"2024-04-25T20:54:29.391559+0000","last_became_peered":"2024-04-25T20:54:29.391559+0000","last_unstale":"2024-04-25T20:57:38.692177+0000","last_undegraded":"2024-04-25T20:57:38.692177+0000","last_fullsized":"2024-04-25T20:57:38.692177+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:43:56.141769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616238+0000","last_change":"2024-04-25T20:54:29.399648+0000","last_active":"2024-04-25T20:57:38.616238+0000","last_peered":"2024-04-25T20:57:38.616238+0000","last_clean":"2024-04-25T20:57:38.616238+0000","last_became_active":"2024-04-25T20:54:29.399254+0000","last_became_peered":"2024-04-25T20:54:29.399254+0000","last_unstale":"2024-04-25T20:57:38.616238+0000","last_undegraded":"2024-04-25T20:57:38.616238+0000","last_fullsized":"2024-04-25T20:57:38.616238+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:21:18.995634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"93'10","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614743+0000","last_change":"2024-04-25T20:55:10.332732+0000","last_active":"2024-04-25T20:57:38.614743+0000","last_peered":"2024-04-25T20:57:38.614743+0000","last_clean":"2024-04-25T20:57:38.614743+0000","last_became_active":"2024-04-25T20:55:10.332197+0000","last_became_peered":"2024-04-25T20:55:10.332197+0000","last_unstale":"2024-04-25T20:57:38.614743+0000","last_undegraded":"2024-04-25T20:57:38.614743+0000","last_fullsized":"2024-04-25T20:57:38.614743+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:28:22.612753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,9],"acting":[6,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691157+0000","last_change":"2024-04-25T20:54:29.390693+0000","last_active":"2024-04-25T20:57:38.691157+0000","last_peered":"2024-04-25T20:57:38.691157+0000","last_clean":"2024-04-25T20:57:38.691157+0000","last_became_active":"2024-04-25T20:54:29.390380+0000","last_became_peered":"2024-04-25T20:54:29.390380+0000","last_unstale":"2024-04-25T20:57:38.691157+0000","last_undegraded":"2024-04-25T20:57:38.691157+0000","last_fullsized":"2024-04-25T20:57:38.691157+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:45:17.284633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691059+0000","last_change":"2024-04-25T20:54:29.395825+0000","last_active":"2024-04-25T20:57:38.691059+0000","last_peered":"2024-04-25T20:57:38.691059+0000","last_clean":"2024-04-25T20:57:38.691059+0000","last_became_active":"2024-04-25T20:54:29.395522+0000","last_became_peered":"2024-04-25T20:54:29.395522+0000","last_unstale":"2024-04-25T20:57:38.691059+0000","last_undegraded":"2024-04-25T20:57:38.691059+0000","last_fullsized":"2024-04-25T20:57:38.691059+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:04:30.745839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691068+0000","last_change":"2024-04-25T20:54:29.391041+0000","last_active":"2024-04-25T20:57:38.691068+0000","last_peered":"2024-04-25T20:57:38.691068+0000","last_clean":"2024-04-25T20:57:38.691068+0000","last_became_active":"2024-04-25T20:54:29.390721+0000","last_became_peered":"2024-04-25T20:54:29.390721+0000","last_unstale":"2024-04-25T20:57:38.691068+0000","last_undegraded":"2024-04-25T20:57:38.691068+0000","last_fullsized":"2024-04-25T20:57:38.691068+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:02:40.582809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191562+0000","last_change":"2024-04-25T20:54:29.396583+0000","last_active":"2024-04-25T20:57:39.191562+0000","last_peered":"2024-04-25T20:57:39.191562+0000","last_clean":"2024-04-25T20:57:39.191562+0000","last_became_active":"2024-04-25T20:54:29.396233+0000","last_became_peered":"2024-04-25T20:54:29.396233+0000","last_unstale":"2024-04-25T20:57:39.191562+0000","last_undegraded":"2024-04-25T20:57:39.191562+0000","last_fullsized":"2024-04-25T20:57:39.191562+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:24:44.033361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.620534+0000","last_change":"2024-04-25T20:54:29.390363+0000","last_active":"2024-04-25T20:57:38.620534+0000","last_peered":"2024-04-25T20:57:38.620534+0000","last_clean":"2024-04-25T20:57:38.620534+0000","last_became_active":"2024-04-25T20:54:29.390040+0000","last_became_peered":"2024-04-25T20:54:29.390040+0000","last_unstale":"2024-04-25T20:57:38.620534+0000","last_undegraded":"2024-04-25T20:57:38.620534+0000","last_fullsized":"2024-04-25T20:57:38.620534+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:55.541075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"93'7","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524590+0000","last_change":"2024-04-25T20:54:29.392748+0000","last_active":"2024-04-25T20:57:38.524590+0000","last_peered":"2024-04-25T20:57:38.524590+0000","last_clean":"2024-04-25T20:57:38.524590+0000","last_became_active":"2024-04-25T20:54:29.392430+0000","last_became_peered":"2024-04-25T20:54:29.392430+0000","last_unstale":"2024-04-25T20:57:38.524590+0000","last_undegraded":"2024-04-25T20:57:38.524590+0000","last_fullsized":"2024-04-25T20:57:38.524590+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:09.624035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'9","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.620316+0000","last_change":"2024-04-25T20:54:26.365407+0000","last_active":"2024-04-25T20:57:38.620316+0000","last_peered":"2024-04-25T20:57:38.620316+0000","last_clean":"2024-04-25T20:57:38.620316+0000","last_became_active":"2024-04-25T20:54:26.364965+0000","last_became_peered":"2024-04-25T20:54:26.364965+0000","last_unstale":"2024-04-25T20:57:38.620316+0000","last_undegraded":"2024-04-25T20:57:38.620316+0000","last_fullsized":"2024-04-25T20:57:38.620316+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:36.311584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523372+0000","last_change":"2024-04-25T20:54:29.394122+0000","last_active":"2024-04-25T20:57:38.523372+0000","last_peered":"2024-04-25T20:57:38.523372+0000","last_clean":"2024-04-25T20:57:38.523372+0000","last_became_active":"2024-04-25T20:54:29.393813+0000","last_became_peered":"2024-04-25T20:54:29.393813+0000","last_unstale":"2024-04-25T20:57:38.523372+0000","last_undegraded":"2024-04-25T20:57:38.523372+0000","last_fullsized":"2024-04-25T20:57:38.523372+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:37:11.336721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523534+0000","last_change":"2024-04-25T20:54:26.549641+0000","last_active":"2024-04-25T20:57:38.523534+0000","last_peered":"2024-04-25T20:57:38.523534+0000","last_clean":"2024-04-25T20:57:38.523534+0000","last_became_active":"2024-04-25T20:54:26.549300+0000","last_became_peered":"2024-04-25T20:54:26.549300+0000","last_unstale":"2024-04-25T20:57:38.523534+0000","last_undegraded":"2024-04-25T20:57:38.523534+0000","last_fullsized":"2024-04-25T20:57:38.523534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:48:35.717614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691859+0000","last_change":"2024-04-25T20:54:29.391261+0000","last_active":"2024-04-25T20:57:38.691859+0000","last_peered":"2024-04-25T20:57:38.691859+0000","last_clean":"2024-04-25T20:57:38.691859+0000","last_became_active":"2024-04-25T20:54:29.390600+0000","last_became_peered":"2024-04-25T20:54:29.390600+0000","last_unstale":"2024-04-25T20:57:38.691859+0000","last_undegraded":"2024-04-25T20:57:38.691859+0000","last_fullsized":"2024-04-25T20:57:38.691859+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:26:55.825049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"91'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622881+0000","last_change":"2024-04-25T20:54:26.364606+0000","last_active":"2024-04-25T20:57:38.622881+0000","last_peered":"2024-04-25T20:57:38.622881+0000","last_clean":"2024-04-25T20:57:38.622881+0000","last_became_active":"2024-04-25T20:54:26.364233+0000","last_became_peered":"2024-04-25T20:54:26.364233+0000","last_unstale":"2024-04-25T20:57:38.622881+0000","last_undegraded":"2024-04-25T20:57:38.622881+0000","last_fullsized":"2024-04-25T20:57:38.622881+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:49.438911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691786+0000","last_change":"2024-04-25T20:54:29.394073+0000","last_active":"2024-04-25T20:57:38.691786+0000","last_peered":"2024-04-25T20:57:38.691786+0000","last_clean":"2024-04-25T20:57:38.691786+0000","last_became_active":"2024-04-25T20:54:29.393745+0000","last_became_peered":"2024-04-25T20:54:29.393745+0000","last_unstale":"2024-04-25T20:57:38.691786+0000","last_undegraded":"2024-04-25T20:57:38.691786+0000","last_fullsized":"2024-04-25T20:57:38.691786+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:47:23.090719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622783+0000","last_change":"2024-04-25T20:54:26.359451+0000","last_active":"2024-04-25T20:57:38.622783+0000","last_peered":"2024-04-25T20:57:38.622783+0000","last_clean":"2024-04-25T20:57:38.622783+0000","last_became_active":"2024-04-25T20:54:26.359093+0000","last_became_peered":"2024-04-25T20:54:26.359093+0000","last_unstale":"2024-04-25T20:57:38.622783+0000","last_undegraded":"2024-04-25T20:57:38.622783+0000","last_fullsized":"2024-04-25T20:57:38.622783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:25:20.344871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"93'15","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616254+0000","last_change":"2024-04-25T20:54:29.399079+0000","last_active":"2024-04-25T20:57:38.616254+0000","last_peered":"2024-04-25T20:57:38.616254+0000","last_clean":"2024-04-25T20:57:38.616254+0000","last_became_active":"2024-04-25T20:54:29.398749+0000","last_became_peered":"2024-04-25T20:54:29.398749+0000","last_unstale":"2024-04-25T20:57:38.616254+0000","last_undegraded":"2024-04-25T20:57:38.616254+0000","last_fullsized":"2024-04-25T20:57:38.616254+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:20:46.762400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'10","reported_seq":223,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:42.621808+0000","last_change":"2024-04-25T20:54:26.362267+0000","last_active":"2024-04-25T20:57:42.621808+0000","last_peered":"2024-04-25T20:57:42.621808+0000","last_clean":"2024-04-25T20:57:42.621808+0000","last_became_active":"2024-04-25T20:54:26.361846+0000","last_became_peered":"2024-04-25T20:54:26.361846+0000","last_unstale":"2024-04-25T20:57:42.621808+0000","last_undegraded":"2024-04-25T20:57:42.621808+0000","last_fullsized":"2024-04-25T20:57:42.621808+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:46:21.216620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2244,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":184,"num_read_kb":6,"num_write":7,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"93'7","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622512+0000","last_change":"2024-04-25T20:54:29.399105+0000","last_active":"2024-04-25T20:57:38.622512+0000","last_peered":"2024-04-25T20:57:38.622512+0000","last_clean":"2024-04-25T20:57:38.622512+0000","last_became_active":"2024-04-25T20:54:29.398770+0000","last_became_peered":"2024-04-25T20:54:29.398770+0000","last_unstale":"2024-04-25T20:57:38.622512+0000","last_undegraded":"2024-04-25T20:57:38.622512+0000","last_fullsized":"2024-04-25T20:57:38.622512+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:32:31.486683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522207+0000","last_change":"2024-04-25T20:54:26.362100+0000","last_active":"2024-04-25T20:57:38.522207+0000","last_peered":"2024-04-25T20:57:38.522207+0000","last_clean":"2024-04-25T20:57:38.522207+0000","last_became_active":"2024-04-25T20:54:26.361637+0000","last_became_peered":"2024-04-25T20:54:26.361637+0000","last_unstale":"2024-04-25T20:57:38.522207+0000","last_undegraded":"2024-04-25T20:57:38.522207+0000","last_fullsized":"2024-04-25T20:57:38.522207+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:23:04.337096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615952+0000","last_change":"2024-04-25T20:54:29.397549+0000","last_active":"2024-04-25T20:57:38.615952+0000","last_peered":"2024-04-25T20:57:38.615952+0000","last_clean":"2024-04-25T20:57:38.615952+0000","last_became_active":"2024-04-25T20:54:29.397240+0000","last_became_peered":"2024-04-25T20:54:29.397240+0000","last_unstale":"2024-04-25T20:57:38.615952+0000","last_undegraded":"2024-04-25T20:57:38.615952+0000","last_fullsized":"2024-04-25T20:57:38.615952+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:41:56.821334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614503+0000","last_change":"2024-04-25T20:54:26.362774+0000","last_active":"2024-04-25T20:57:38.614503+0000","last_peered":"2024-04-25T20:57:38.614503+0000","last_clean":"2024-04-25T20:57:38.614503+0000","last_became_active":"2024-04-25T20:54:26.362412+0000","last_became_peered":"2024-04-25T20:54:26.362412+0000","last_unstale":"2024-04-25T20:57:38.614503+0000","last_undegraded":"2024-04-25T20:57:38.614503+0000","last_fullsized":"2024-04-25T20:57:38.614503+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:27.167674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522962+0000","last_change":"2024-04-25T20:54:29.387203+0000","last_active":"2024-04-25T20:57:38.522962+0000","last_peered":"2024-04-25T20:57:38.522962+0000","last_clean":"2024-04-25T20:57:38.522962+0000","last_became_active":"2024-04-25T20:54:29.386856+0000","last_became_peered":"2024-04-25T20:54:29.386856+0000","last_unstale":"2024-04-25T20:57:38.522962+0000","last_undegraded":"2024-04-25T20:57:38.522962+0000","last_fullsized":"2024-04-25T20:57:38.522962+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:55.288659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"91'1","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615783+0000","last_change":"2024-04-25T20:54:26.362084+0000","last_active":"2024-04-25T20:57:38.615783+0000","last_peered":"2024-04-25T20:57:38.615783+0000","last_clean":"2024-04-25T20:57:38.615783+0000","last_became_active":"2024-04-25T20:54:26.361683+0000","last_became_peered":"2024-04-25T20:54:26.361683+0000","last_unstale":"2024-04-25T20:57:38.615783+0000","last_undegraded":"2024-04-25T20:57:38.615783+0000","last_fullsized":"2024-04-25T20:57:38.615783+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:51:54.347937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.619924+0000","last_change":"2024-04-25T20:54:29.385960+0000","last_active":"2024-04-25T20:57:38.619924+0000","last_peered":"2024-04-25T20:57:38.619924+0000","last_clean":"2024-04-25T20:57:38.619924+0000","last_became_active":"2024-04-25T20:54:29.385665+0000","last_became_peered":"2024-04-25T20:54:29.385665+0000","last_unstale":"2024-04-25T20:57:38.619924+0000","last_undegraded":"2024-04-25T20:57:38.619924+0000","last_fullsized":"2024-04-25T20:57:38.619924+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:20:06.907328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"93'5","reported_seq":211,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:42.621318+0000","last_change":"2024-04-25T20:54:26.365008+0000","last_active":"2024-04-25T20:57:42.621318+0000","last_peered":"2024-04-25T20:57:42.621318+0000","last_clean":"2024-04-25T20:57:42.621318+0000","last_became_active":"2024-04-25T20:54:26.364579+0000","last_became_peered":"2024-04-25T20:54:26.364579+0000","last_unstale":"2024-04-25T20:57:42.621318+0000","last_undegraded":"2024-04-25T20:57:42.621318+0000","last_fullsized":"2024-04-25T20:57:42.621318+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:23:14.465250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691858+0000","last_change":"2024-04-25T20:54:29.390276+0000","last_active":"2024-04-25T20:57:38.691858+0000","last_peered":"2024-04-25T20:57:38.691858+0000","last_clean":"2024-04-25T20:57:38.691858+0000","last_became_active":"2024-04-25T20:54:29.389901+0000","last_became_peered":"2024-04-25T20:54:29.389901+0000","last_unstale":"2024-04-25T20:57:38.691858+0000","last_undegraded":"2024-04-25T20:57:38.691858+0000","last_fullsized":"2024-04-25T20:57:38.691858+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:35.608004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"91'2","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692223+0000","last_change":"2024-04-25T20:54:26.362781+0000","last_active":"2024-04-25T20:57:38.692223+0000","last_peered":"2024-04-25T20:57:38.692223+0000","last_clean":"2024-04-25T20:57:38.692223+0000","last_became_active":"2024-04-25T20:54:26.362417+0000","last_became_peered":"2024-04-25T20:54:26.362417+0000","last_unstale":"2024-04-25T20:57:38.692223+0000","last_undegraded":"2024-04-25T20:57:38.692223+0000","last_fullsized":"2024-04-25T20:57:38.692223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:43:25.097916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"93'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691875+0000","last_change":"2024-04-25T20:55:10.334414+0000","last_active":"2024-04-25T20:57:38.691875+0000","last_peered":"2024-04-25T20:57:38.691875+0000","last_clean":"2024-04-25T20:57:38.691875+0000","last_became_active":"2024-04-25T20:55:10.334022+0000","last_became_peered":"2024-04-25T20:55:10.334022+0000","last_unstale":"2024-04-25T20:57:38.691875+0000","last_undegraded":"2024-04-25T20:57:38.691875+0000","last_fullsized":"2024-04-25T20:57:38.691875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:37:32.914876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"91'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615739+0000","last_change":"2024-04-25T20:54:26.550580+0000","last_active":"2024-04-25T20:57:38.615739+0000","last_peered":"2024-04-25T20:57:38.615739+0000","last_clean":"2024-04-25T20:57:38.615739+0000","last_became_active":"2024-04-25T20:54:26.550090+0000","last_became_peered":"2024-04-25T20:54:26.550090+0000","last_unstale":"2024-04-25T20:57:38.615739+0000","last_undegraded":"2024-04-25T20:57:38.615739+0000","last_fullsized":"2024-04-25T20:57:38.615739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:51:42.379374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691880+0000","last_change":"2024-04-25T20:54:29.400744+0000","last_active":"2024-04-25T20:57:38.691880+0000","last_peered":"2024-04-25T20:57:38.691880+0000","last_clean":"2024-04-25T20:57:38.691880+0000","last_became_active":"2024-04-25T20:54:29.400395+0000","last_became_peered":"2024-04-25T20:54:29.400395+0000","last_unstale":"2024-04-25T20:57:38.691880+0000","last_undegraded":"2024-04-25T20:57:38.691880+0000","last_fullsized":"2024-04-25T20:57:38.691880+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:55:56.103379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'39","reported_seq":437,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:43.623100+0000","last_change":"2024-04-25T20:54:26.365517+0000","last_active":"2024-04-25T20:57:43.623100+0000","last_peered":"2024-04-25T20:57:43.623100+0000","last_clean":"2024-04-25T20:57:43.623100+0000","last_became_active":"2024-04-25T20:54:26.365105+0000","last_became_peered":"2024-04-25T20:54:26.365105+0000","last_unstale":"2024-04-25T20:57:43.623100+0000","last_undegraded":"2024-04-25T20:57:43.623100+0000","last_fullsized":"2024-04-25T20:57:43.623100+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:32.409796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69970,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":370,"num_read_kb":254,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622687+0000","last_change":"2024-04-25T20:54:29.387883+0000","last_active":"2024-04-25T20:57:38.622687+0000","last_peered":"2024-04-25T20:57:38.622687+0000","last_clean":"2024-04-25T20:57:38.622687+0000","last_became_active":"2024-04-25T20:54:29.387563+0000","last_became_peered":"2024-04-25T20:54:29.387563+0000","last_unstale":"2024-04-25T20:57:38.622687+0000","last_undegraded":"2024-04-25T20:57:38.622687+0000","last_fullsized":"2024-04-25T20:57:38.622687+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:22:40.540600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524419+0000","last_change":"2024-04-25T20:54:26.363389+0000","last_active":"2024-04-25T20:57:38.524419+0000","last_peered":"2024-04-25T20:57:38.524419+0000","last_clean":"2024-04-25T20:57:38.524419+0000","last_became_active":"2024-04-25T20:54:26.362941+0000","last_became_peered":"2024-04-25T20:54:26.362941+0000","last_unstale":"2024-04-25T20:57:38.524419+0000","last_undegraded":"2024-04-25T20:57:38.524419+0000","last_fullsized":"2024-04-25T20:57:38.524419+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:45:42.931863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614481+0000","last_change":"2024-04-25T20:54:29.395028+0000","last_active":"2024-04-25T20:57:38.614481+0000","last_peered":"2024-04-25T20:57:38.614481+0000","last_clean":"2024-04-25T20:57:38.614481+0000","last_became_active":"2024-04-25T20:54:29.393497+0000","last_became_peered":"2024-04-25T20:54:29.393497+0000","last_unstale":"2024-04-25T20:57:38.614481+0000","last_undegraded":"2024-04-25T20:57:38.614481+0000","last_fullsized":"2024-04-25T20:57:38.614481+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:21:10.759881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"91'3","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616027+0000","last_change":"2024-04-25T20:54:26.365278+0000","last_active":"2024-04-25T20:57:38.616027+0000","last_peered":"2024-04-25T20:57:38.616027+0000","last_clean":"2024-04-25T20:57:38.616027+0000","last_became_active":"2024-04-25T20:54:26.364900+0000","last_became_peered":"2024-04-25T20:54:26.364900+0000","last_unstale":"2024-04-25T20:57:38.616027+0000","last_undegraded":"2024-04-25T20:57:38.616027+0000","last_fullsized":"2024-04-25T20:57:38.616027+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:55:18.476661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"93'15","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691679+0000","last_change":"2024-04-25T20:54:29.396226+0000","last_active":"2024-04-25T20:57:38.691679+0000","last_peered":"2024-04-25T20:57:38.691679+0000","last_clean":"2024-04-25T20:57:38.691679+0000","last_became_active":"2024-04-25T20:54:29.395689+0000","last_became_peered":"2024-04-25T20:54:29.395689+0000","last_unstale":"2024-04-25T20:57:38.691679+0000","last_undegraded":"2024-04-25T20:57:38.691679+0000","last_fullsized":"2024-04-25T20:57:38.691679+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:01.739919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'4","reported_seq":213,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:42.621865+0000","last_change":"2024-04-25T20:54:26.359558+0000","last_active":"2024-04-25T20:57:42.621865+0000","last_peered":"2024-04-25T20:57:42.621865+0000","last_clean":"2024-04-25T20:57:42.621865+0000","last_became_active":"2024-04-25T20:54:26.359111+0000","last_became_peered":"2024-04-25T20:54:26.359111+0000","last_unstale":"2024-04-25T20:57:42.621865+0000","last_undegraded":"2024-04-25T20:57:42.621865+0000","last_fullsized":"2024-04-25T20:57:42.621865+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:26:58.603656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":182,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691075+0000","last_change":"2024-04-25T20:54:29.401891+0000","last_active":"2024-04-25T20:57:38.691075+0000","last_peered":"2024-04-25T20:57:38.691075+0000","last_clean":"2024-04-25T20:57:38.691075+0000","last_became_active":"2024-04-25T20:54:29.400794+0000","last_became_peered":"2024-04-25T20:54:29.400794+0000","last_unstale":"2024-04-25T20:57:38.691075+0000","last_undegraded":"2024-04-25T20:57:38.691075+0000","last_fullsized":"2024-04-25T20:57:38.691075+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:36.299525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"91'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622432+0000","last_change":"2024-04-25T20:54:26.550146+0000","last_active":"2024-04-25T20:57:38.622432+0000","last_peered":"2024-04-25T20:57:38.622432+0000","last_clean":"2024-04-25T20:57:38.622432+0000","last_became_active":"2024-04-25T20:54:26.549460+0000","last_became_peered":"2024-04-25T20:54:26.549460+0000","last_unstale":"2024-04-25T20:57:38.622432+0000","last_undegraded":"2024-04-25T20:57:38.622432+0000","last_fullsized":"2024-04-25T20:57:38.622432+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:16:44.894187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615853+0000","last_change":"2024-04-25T20:54:29.399825+0000","last_active":"2024-04-25T20:57:38.615853+0000","last_peered":"2024-04-25T20:57:38.615853+0000","last_clean":"2024-04-25T20:57:38.615853+0000","last_became_active":"2024-04-25T20:54:29.399516+0000","last_became_peered":"2024-04-25T20:54:29.399516+0000","last_unstale":"2024-04-25T20:57:38.615853+0000","last_undegraded":"2024-04-25T20:57:38.615853+0000","last_fullsized":"2024-04-25T20:57:38.615853+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:27:43.588617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"91'2","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622920+0000","last_change":"2024-04-25T20:54:26.360795+0000","last_active":"2024-04-25T20:57:38.622920+0000","last_peered":"2024-04-25T20:57:38.622920+0000","last_clean":"2024-04-25T20:57:38.622920+0000","last_became_active":"2024-04-25T20:54:26.360481+0000","last_became_peered":"2024-04-25T20:54:26.360481+0000","last_unstale":"2024-04-25T20:57:38.622920+0000","last_undegraded":"2024-04-25T20:57:38.622920+0000","last_fullsized":"2024-04-25T20:57:38.622920+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:33:01.477626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191264+0000","last_change":"2024-04-25T20:54:29.398306+0000","last_active":"2024-04-25T20:57:39.191264+0000","last_peered":"2024-04-25T20:57:39.191264+0000","last_clean":"2024-04-25T20:57:39.191264+0000","last_became_active":"2024-04-25T20:54:29.397956+0000","last_became_peered":"2024-04-25T20:54:29.397956+0000","last_unstale":"2024-04-25T20:57:39.191264+0000","last_undegraded":"2024-04-25T20:57:39.191264+0000","last_fullsized":"2024-04-25T20:57:39.191264+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:12:39.892955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.619738+0000","last_change":"2024-04-25T20:54:26.549533+0000","last_active":"2024-04-25T20:57:38.619738+0000","last_peered":"2024-04-25T20:57:38.619738+0000","last_clean":"2024-04-25T20:57:38.619738+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:57:38.619738+0000","last_undegraded":"2024-04-25T20:57:38.619738+0000","last_fullsized":"2024-04-25T20:57:38.619738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:23:02.657125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522451+0000","last_change":"2024-04-25T20:54:29.397346+0000","last_active":"2024-04-25T20:57:38.522451+0000","last_peered":"2024-04-25T20:57:38.522451+0000","last_clean":"2024-04-25T20:57:38.522451+0000","last_became_active":"2024-04-25T20:54:29.396927+0000","last_became_peered":"2024-04-25T20:54:29.396927+0000","last_unstale":"2024-04-25T20:57:38.522451+0000","last_undegraded":"2024-04-25T20:57:38.522451+0000","last_fullsized":"2024-04-25T20:57:38.522451+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:06:05.529821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"91'1","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691667+0000","last_change":"2024-04-25T20:54:26.361699+0000","last_active":"2024-04-25T20:57:38.691667+0000","last_peered":"2024-04-25T20:57:38.691667+0000","last_clean":"2024-04-25T20:57:38.691667+0000","last_became_active":"2024-04-25T20:54:26.361323+0000","last_became_peered":"2024-04-25T20:54:26.361323+0000","last_unstale":"2024-04-25T20:57:38.691667+0000","last_undegraded":"2024-04-25T20:57:38.691667+0000","last_fullsized":"2024-04-25T20:57:38.691667+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:42:08.657311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"93'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522266+0000","last_change":"2024-04-25T20:55:10.332432+0000","last_active":"2024-04-25T20:57:38.522266+0000","last_peered":"2024-04-25T20:57:38.522266+0000","last_clean":"2024-04-25T20:57:38.522266+0000","last_became_active":"2024-04-25T20:55:10.332033+0000","last_became_peered":"2024-04-25T20:55:10.332033+0000","last_unstale":"2024-04-25T20:57:38.522266+0000","last_undegraded":"2024-04-25T20:57:38.522266+0000","last_fullsized":"2024-04-25T20:57:38.522266+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:28:34.930705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616167+0000","last_change":"2024-04-25T20:54:29.396344+0000","last_active":"2024-04-25T20:57:38.616167+0000","last_peered":"2024-04-25T20:57:38.616167+0000","last_clean":"2024-04-25T20:57:38.616167+0000","last_became_active":"2024-04-25T20:54:29.396033+0000","last_became_peered":"2024-04-25T20:54:29.396033+0000","last_unstale":"2024-04-25T20:57:38.616167+0000","last_undegraded":"2024-04-25T20:57:38.616167+0000","last_fullsized":"2024-04-25T20:57:38.616167+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:01:26.103442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'5","reported_seq":212,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:42.621458+0000","last_change":"2024-04-25T20:54:26.549512+0000","last_active":"2024-04-25T20:57:42.621458+0000","last_peered":"2024-04-25T20:57:42.621458+0000","last_clean":"2024-04-25T20:57:42.621458+0000","last_became_active":"2024-04-25T20:54:26.549158+0000","last_became_peered":"2024-04-25T20:54:26.549158+0000","last_unstale":"2024-04-25T20:57:42.621458+0000","last_undegraded":"2024-04-25T20:57:42.621458+0000","last_fullsized":"2024-04-25T20:57:42.621458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:05:31.777057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523216+0000","last_change":"2024-04-25T20:54:29.394171+0000","last_active":"2024-04-25T20:57:38.523216+0000","last_peered":"2024-04-25T20:57:38.523216+0000","last_clean":"2024-04-25T20:57:38.523216+0000","last_became_active":"2024-04-25T20:54:29.393846+0000","last_became_peered":"2024-04-25T20:54:29.393846+0000","last_unstale":"2024-04-25T20:57:38.523216+0000","last_undegraded":"2024-04-25T20:57:38.523216+0000","last_fullsized":"2024-04-25T20:57:38.523216+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:12:11.372114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"91'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616224+0000","last_change":"2024-04-25T20:54:26.362971+0000","last_active":"2024-04-25T20:57:38.616224+0000","last_peered":"2024-04-25T20:57:38.616224+0000","last_clean":"2024-04-25T20:57:38.616224+0000","last_became_active":"2024-04-25T20:54:26.362544+0000","last_became_peered":"2024-04-25T20:54:26.362544+0000","last_unstale":"2024-04-25T20:57:38.616224+0000","last_undegraded":"2024-04-25T20:57:38.616224+0000","last_fullsized":"2024-04-25T20:57:38.616224+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:30:59.398066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":408,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:40.623801+0000","last_change":"2024-04-25T20:53:01.015126+0000","last_active":"2024-04-25T20:57:40.623801+0000","last_peered":"2024-04-25T20:57:40.623801+0000","last_clean":"2024-04-25T20:57:40.623801+0000","last_became_active":"2024-04-25T20:53:01.014729+0000","last_became_peered":"2024-04-25T20:53:01.014729+0000","last_unstale":"2024-04-25T20:57:40.623801+0000","last_undegraded":"2024-04-25T20:57:40.623801+0000","last_fullsized":"2024-04-25T20:57:40.623801+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:49:37.859710+0000","last_clean_scrub_stamp":"2024-04-25T20:49:37.859710+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:11:51.601723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"93'5","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614336+0000","last_change":"2024-04-25T20:54:29.398595+0000","last_active":"2024-04-25T20:57:38.614336+0000","last_peered":"2024-04-25T20:57:38.614336+0000","last_clean":"2024-04-25T20:57:38.614336+0000","last_became_active":"2024-04-25T20:54:29.398295+0000","last_became_peered":"2024-04-25T20:54:29.398295+0000","last_unstale":"2024-04-25T20:57:38.614336+0000","last_undegraded":"2024-04-25T20:57:38.614336+0000","last_fullsized":"2024-04-25T20:57:38.614336+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:40:17.989985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691380+0000","last_change":"2024-04-25T20:54:26.361803+0000","last_active":"2024-04-25T20:57:38.691380+0000","last_peered":"2024-04-25T20:57:38.691380+0000","last_clean":"2024-04-25T20:57:38.691380+0000","last_became_active":"2024-04-25T20:54:26.361367+0000","last_became_peered":"2024-04-25T20:54:26.361367+0000","last_unstale":"2024-04-25T20:57:38.691380+0000","last_undegraded":"2024-04-25T20:57:38.691380+0000","last_fullsized":"2024-04-25T20:57:38.691380+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:43.937365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"91'1","reported_seq":18,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191351+0000","last_change":"2024-04-25T20:55:10.337450+0000","last_active":"2024-04-25T20:57:39.191351+0000","last_peered":"2024-04-25T20:57:39.191351+0000","last_clean":"2024-04-25T20:57:39.191351+0000","last_became_active":"2024-04-25T20:55:10.332821+0000","last_became_peered":"2024-04-25T20:55:10.332821+0000","last_unstale":"2024-04-25T20:57:39.191351+0000","last_undegraded":"2024-04-25T20:57:39.191351+0000","last_fullsized":"2024-04-25T20:57:39.191351+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:15:55.070702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"93'9","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614931+0000","last_change":"2024-04-25T20:54:29.397253+0000","last_active":"2024-04-25T20:57:38.614931+0000","last_peered":"2024-04-25T20:57:38.614931+0000","last_clean":"2024-04-25T20:57:38.614931+0000","last_became_active":"2024-04-25T20:54:29.396892+0000","last_became_peered":"2024-04-25T20:54:29.396892+0000","last_unstale":"2024-04-25T20:57:38.614931+0000","last_undegraded":"2024-04-25T20:57:38.614931+0000","last_fullsized":"2024-04-25T20:57:38.614931+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:17:40.945659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'102","reported_seq":223,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:41.123548+0000","last_change":"2024-04-25T20:54:26.362551+0000","last_active":"2024-04-25T20:57:41.123548+0000","last_peered":"2024-04-25T20:57:41.123548+0000","last_clean":"2024-04-25T20:57:41.123548+0000","last_became_active":"2024-04-25T20:54:26.362212+0000","last_became_peered":"2024-04-25T20:54:26.362212+0000","last_unstale":"2024-04-25T20:57:41.123548+0000","last_undegraded":"2024-04-25T20:57:41.123548+0000","last_fullsized":"2024-04-25T20:57:41.123548+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:13:50.222715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":188,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523457+0000","last_change":"2024-04-25T20:54:29.387488+0000","last_active":"2024-04-25T20:57:38.523457+0000","last_peered":"2024-04-25T20:57:38.523457+0000","last_clean":"2024-04-25T20:57:38.523457+0000","last_became_active":"2024-04-25T20:54:29.387064+0000","last_became_peered":"2024-04-25T20:54:29.387064+0000","last_unstale":"2024-04-25T20:57:38.523457+0000","last_undegraded":"2024-04-25T20:57:38.523457+0000","last_fullsized":"2024-04-25T20:57:38.523457+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:03:48.212247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"91'4","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.616365+0000","last_change":"2024-04-25T20:54:26.689091+0000","last_active":"2024-04-25T20:57:38.616365+0000","last_peered":"2024-04-25T20:57:38.616365+0000","last_clean":"2024-04-25T20:57:38.616365+0000","last_became_active":"2024-04-25T20:54:26.688724+0000","last_became_peered":"2024-04-25T20:54:26.688724+0000","last_unstale":"2024-04-25T20:57:38.616365+0000","last_undegraded":"2024-04-25T20:57:38.616365+0000","last_fullsized":"2024-04-25T20:57:38.616365+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:08:04.586424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691756+0000","last_change":"2024-04-25T20:54:29.398450+0000","last_active":"2024-04-25T20:57:38.691756+0000","last_peered":"2024-04-25T20:57:38.691756+0000","last_clean":"2024-04-25T20:57:38.691756+0000","last_became_active":"2024-04-25T20:54:29.398139+0000","last_became_peered":"2024-04-25T20:54:29.398139+0000","last_unstale":"2024-04-25T20:57:38.691756+0000","last_undegraded":"2024-04-25T20:57:38.691756+0000","last_fullsized":"2024-04-25T20:57:38.691756+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:29.240410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691035+0000","last_change":"2024-04-25T20:54:26.549358+0000","last_active":"2024-04-25T20:57:38.691035+0000","last_peered":"2024-04-25T20:57:38.691035+0000","last_clean":"2024-04-25T20:57:38.691035+0000","last_became_active":"2024-04-25T20:54:26.549049+0000","last_became_peered":"2024-04-25T20:54:26.549049+0000","last_unstale":"2024-04-25T20:57:38.691035+0000","last_undegraded":"2024-04-25T20:57:38.691035+0000","last_fullsized":"2024-04-25T20:57:38.691035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:25:11.609865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"93'2","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622483+0000","last_change":"2024-04-25T20:54:29.389749+0000","last_active":"2024-04-25T20:57:38.622483+0000","last_peered":"2024-04-25T20:57:38.622483+0000","last_clean":"2024-04-25T20:57:38.622483+0000","last_became_active":"2024-04-25T20:54:29.389388+0000","last_became_peered":"2024-04-25T20:54:29.389388+0000","last_unstale":"2024-04-25T20:57:38.622483+0000","last_undegraded":"2024-04-25T20:57:38.622483+0000","last_fullsized":"2024-04-25T20:57:38.622483+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:54:38.105725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"91'3","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523034+0000","last_change":"2024-04-25T20:54:26.364278+0000","last_active":"2024-04-25T20:57:38.523034+0000","last_peered":"2024-04-25T20:57:38.523034+0000","last_clean":"2024-04-25T20:57:38.523034+0000","last_became_active":"2024-04-25T20:54:26.363933+0000","last_became_peered":"2024-04-25T20:54:26.363933+0000","last_unstale":"2024-04-25T20:57:38.523034+0000","last_undegraded":"2024-04-25T20:57:38.523034+0000","last_fullsized":"2024-04-25T20:57:38.523034+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:06:00.805752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191000+0000","last_change":"2024-04-25T20:54:29.392849+0000","last_active":"2024-04-25T20:57:39.191000+0000","last_peered":"2024-04-25T20:57:39.191000+0000","last_clean":"2024-04-25T20:57:39.191000+0000","last_became_active":"2024-04-25T20:54:29.392509+0000","last_became_peered":"2024-04-25T20:54:29.392509+0000","last_unstale":"2024-04-25T20:57:39.191000+0000","last_undegraded":"2024-04-25T20:57:39.191000+0000","last_fullsized":"2024-04-25T20:57:39.191000+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:29:09.946646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'3","reported_seq":20,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.345325+0000","last_change":"2024-04-25T20:55:10.339729+0000","last_active":"2024-04-25T20:57:39.345325+0000","last_peered":"2024-04-25T20:57:39.345325+0000","last_clean":"2024-04-25T20:57:39.345325+0000","last_became_active":"2024-04-25T20:55:10.333756+0000","last_became_peered":"2024-04-25T20:55:10.333756+0000","last_unstale":"2024-04-25T20:57:39.345325+0000","last_undegraded":"2024-04-25T20:57:39.345325+0000","last_fullsized":"2024-04-25T20:57:39.345325+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:18:48.806227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523332+0000","last_change":"2024-04-25T20:54:29.388077+0000","last_active":"2024-04-25T20:57:38.523332+0000","last_peered":"2024-04-25T20:57:38.523332+0000","last_clean":"2024-04-25T20:57:38.523332+0000","last_became_active":"2024-04-25T20:54:29.387806+0000","last_became_peered":"2024-04-25T20:54:29.387806+0000","last_unstale":"2024-04-25T20:57:38.523332+0000","last_undegraded":"2024-04-25T20:57:38.523332+0000","last_fullsized":"2024-04-25T20:57:38.523332+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:14:29.764894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"91'4","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.622600+0000","last_change":"2024-04-25T20:54:26.360629+0000","last_active":"2024-04-25T20:57:38.622600+0000","last_peered":"2024-04-25T20:57:38.622600+0000","last_clean":"2024-04-25T20:57:38.622600+0000","last_became_active":"2024-04-25T20:54:26.359930+0000","last_became_peered":"2024-04-25T20:54:26.359930+0000","last_unstale":"2024-04-25T20:57:38.622600+0000","last_undegraded":"2024-04-25T20:57:38.622600+0000","last_fullsized":"2024-04-25T20:57:38.622600+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:30.238613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"93'5","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691778+0000","last_change":"2024-04-25T20:54:29.390350+0000","last_active":"2024-04-25T20:57:38.691778+0000","last_peered":"2024-04-25T20:57:38.691778+0000","last_clean":"2024-04-25T20:57:38.691778+0000","last_became_active":"2024-04-25T20:54:29.390012+0000","last_became_peered":"2024-04-25T20:54:29.390012+0000","last_unstale":"2024-04-25T20:57:38.691778+0000","last_undegraded":"2024-04-25T20:57:38.691778+0000","last_fullsized":"2024-04-25T20:57:38.691778+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:23:01.710860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'3","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524689+0000","last_change":"2024-04-25T20:54:26.689151+0000","last_active":"2024-04-25T20:57:38.524689+0000","last_peered":"2024-04-25T20:57:38.524689+0000","last_clean":"2024-04-25T20:57:38.524689+0000","last_became_active":"2024-04-25T20:54:26.688722+0000","last_became_peered":"2024-04-25T20:54:26.688722+0000","last_unstale":"2024-04-25T20:57:38.524689+0000","last_undegraded":"2024-04-25T20:57:38.524689+0000","last_fullsized":"2024-04-25T20:57:38.524689+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:04:55.553926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692261+0000","last_change":"2024-04-25T20:54:29.392663+0000","last_active":"2024-04-25T20:57:38.692261+0000","last_peered":"2024-04-25T20:57:38.692261+0000","last_clean":"2024-04-25T20:57:38.692261+0000","last_became_active":"2024-04-25T20:54:29.392351+0000","last_became_peered":"2024-04-25T20:54:29.392351+0000","last_unstale":"2024-04-25T20:57:38.692261+0000","last_undegraded":"2024-04-25T20:57:38.692261+0000","last_fullsized":"2024-04-25T20:57:38.692261+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:36:08.127785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'65","reported_seq":125,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:40.737212+0000","last_change":"2024-04-25T20:54:26.359471+0000","last_active":"2024-04-25T20:57:40.737212+0000","last_peered":"2024-04-25T20:57:40.737212+0000","last_clean":"2024-04-25T20:57:40.737212+0000","last_became_active":"2024-04-25T20:54:26.359084+0000","last_became_peered":"2024-04-25T20:54:26.359084+0000","last_unstale":"2024-04-25T20:57:40.737212+0000","last_undegraded":"2024-04-25T20:57:40.737212+0000","last_fullsized":"2024-04-25T20:57:40.737212+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:17:47.085578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691515+0000","last_change":"2024-04-25T20:54:29.401090+0000","last_active":"2024-04-25T20:57:38.691515+0000","last_peered":"2024-04-25T20:57:38.691515+0000","last_clean":"2024-04-25T20:57:38.691515+0000","last_became_active":"2024-04-25T20:54:29.400762+0000","last_became_peered":"2024-04-25T20:54:29.400762+0000","last_unstale":"2024-04-25T20:57:38.691515+0000","last_undegraded":"2024-04-25T20:57:38.691515+0000","last_fullsized":"2024-04-25T20:57:38.691515+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:45.061310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"91'2","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614981+0000","last_change":"2024-04-25T20:54:26.362923+0000","last_active":"2024-04-25T20:57:38.614981+0000","last_peered":"2024-04-25T20:57:38.614981+0000","last_clean":"2024-04-25T20:57:38.614981+0000","last_became_active":"2024-04-25T20:54:26.362607+0000","last_became_peered":"2024-04-25T20:54:26.362607+0000","last_unstale":"2024-04-25T20:57:38.614981+0000","last_undegraded":"2024-04-25T20:57:38.614981+0000","last_fullsized":"2024-04-25T20:57:38.614981+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:25.338574+0000","last_clean_scrub_stamp":"2024-04-25T20:54:25.338574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:56:21.169519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.692338+0000","last_change":"2024-04-25T20:54:29.386514+0000","last_active":"2024-04-25T20:57:38.692338+0000","last_peered":"2024-04-25T20:57:38.692338+0000","last_clean":"2024-04-25T20:57:38.692338+0000","last_became_active":"2024-04-25T20:54:29.386135+0000","last_became_peered":"2024-04-25T20:54:29.386135+0000","last_unstale":"2024-04-25T20:57:38.692338+0000","last_undegraded":"2024-04-25T20:57:38.692338+0000","last_fullsized":"2024-04-25T20:57:38.692338+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:45:29.260309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"93'9","reported_seq":30,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191053+0000","last_change":"2024-04-25T20:54:29.398469+0000","last_active":"2024-04-25T20:57:39.191053+0000","last_peered":"2024-04-25T20:57:39.191053+0000","last_clean":"2024-04-25T20:57:39.191053+0000","last_became_active":"2024-04-25T20:54:29.398090+0000","last_became_peered":"2024-04-25T20:54:29.398090+0000","last_unstale":"2024-04-25T20:57:39.191053+0000","last_undegraded":"2024-04-25T20:57:39.191053+0000","last_fullsized":"2024-04-25T20:57:39.191053+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:46:22.979768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"93'12","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524295+0000","last_change":"2024-04-25T20:54:29.396335+0000","last_active":"2024-04-25T20:57:38.524295+0000","last_peered":"2024-04-25T20:57:38.524295+0000","last_clean":"2024-04-25T20:57:38.524295+0000","last_became_active":"2024-04-25T20:54:29.395955+0000","last_became_peered":"2024-04-25T20:54:29.395955+0000","last_unstale":"2024-04-25T20:57:38.524295+0000","last_undegraded":"2024-04-25T20:57:38.524295+0000","last_fullsized":"2024-04-25T20:57:38.524295+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:49:17.059850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"93'8","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524391+0000","last_change":"2024-04-25T20:54:29.395439+0000","last_active":"2024-04-25T20:57:38.524391+0000","last_peered":"2024-04-25T20:57:38.524391+0000","last_clean":"2024-04-25T20:57:38.524391+0000","last_became_active":"2024-04-25T20:54:29.395105+0000","last_became_peered":"2024-04-25T20:54:29.395105+0000","last_unstale":"2024-04-25T20:57:38.524391+0000","last_undegraded":"2024-04-25T20:57:38.524391+0000","last_fullsized":"2024-04-25T20:57:38.524391+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:53.156374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"93'7","reported_seq":28,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691369+0000","last_change":"2024-04-25T20:54:29.398483+0000","last_active":"2024-04-25T20:57:38.691369+0000","last_peered":"2024-04-25T20:57:38.691369+0000","last_clean":"2024-04-25T20:57:38.691369+0000","last_became_active":"2024-04-25T20:54:29.398159+0000","last_became_peered":"2024-04-25T20:54:29.398159+0000","last_unstale":"2024-04-25T20:57:38.691369+0000","last_undegraded":"2024-04-25T20:57:38.691369+0000","last_fullsized":"2024-04-25T20:57:38.691369+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:06:48.254989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"93'4","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614940+0000","last_change":"2024-04-25T20:55:10.332688+0000","last_active":"2024-04-25T20:57:38.614940+0000","last_peered":"2024-04-25T20:57:38.614940+0000","last_clean":"2024-04-25T20:57:38.614940+0000","last_became_active":"2024-04-25T20:55:10.332207+0000","last_became_peered":"2024-04-25T20:55:10.332207+0000","last_unstale":"2024-04-25T20:57:38.614940+0000","last_undegraded":"2024-04-25T20:57:38.614940+0000","last_fullsized":"2024-04-25T20:57:38.614940+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:11:45.965820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"93'6","reported_seq":27,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191270+0000","last_change":"2024-04-25T20:54:29.399852+0000","last_active":"2024-04-25T20:57:39.191270+0000","last_peered":"2024-04-25T20:57:39.191270+0000","last_clean":"2024-04-25T20:57:39.191270+0000","last_became_active":"2024-04-25T20:54:29.399552+0000","last_became_peered":"2024-04-25T20:54:29.399552+0000","last_unstale":"2024-04-25T20:57:39.191270+0000","last_undegraded":"2024-04-25T20:57:39.191270+0000","last_fullsized":"2024-04-25T20:57:39.191270+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:51.979911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"93'15","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.524541+0000","last_change":"2024-04-25T20:54:29.391022+0000","last_active":"2024-04-25T20:57:38.524541+0000","last_peered":"2024-04-25T20:57:38.524541+0000","last_clean":"2024-04-25T20:57:38.524541+0000","last_became_active":"2024-04-25T20:54:29.390683+0000","last_became_peered":"2024-04-25T20:54:29.390683+0000","last_unstale":"2024-04-25T20:57:38.524541+0000","last_undegraded":"2024-04-25T20:57:38.524541+0000","last_fullsized":"2024-04-25T20:57:38.524541+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:33.203729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"93'2","reported_seq":23,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.523416+0000","last_change":"2024-04-25T20:54:29.395478+0000","last_active":"2024-04-25T20:57:38.523416+0000","last_peered":"2024-04-25T20:57:38.523416+0000","last_clean":"2024-04-25T20:57:38.523416+0000","last_became_active":"2024-04-25T20:54:29.395138+0000","last_became_peered":"2024-04-25T20:54:29.395138+0000","last_unstale":"2024-04-25T20:57:38.523416+0000","last_undegraded":"2024-04-25T20:57:38.523416+0000","last_fullsized":"2024-04-25T20:57:38.523416+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:54.393697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"93'6","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.615843+0000","last_change":"2024-04-25T20:54:29.391779+0000","last_active":"2024-04-25T20:57:38.615843+0000","last_peered":"2024-04-25T20:57:38.615843+0000","last_clean":"2024-04-25T20:57:38.615843+0000","last_became_active":"2024-04-25T20:54:29.390928+0000","last_became_peered":"2024-04-25T20:54:29.390928+0000","last_unstale":"2024-04-25T20:57:38.615843+0000","last_undegraded":"2024-04-25T20:57:38.615843+0000","last_fullsized":"2024-04-25T20:57:38.615843+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:27:59.860533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.522987+0000","last_change":"2024-04-25T20:54:29.392315+0000","last_active":"2024-04-25T20:57:38.522987+0000","last_peered":"2024-04-25T20:57:38.522987+0000","last_clean":"2024-04-25T20:57:38.522987+0000","last_became_active":"2024-04-25T20:54:29.392003+0000","last_became_peered":"2024-04-25T20:54:29.392003+0000","last_unstale":"2024-04-25T20:57:38.522987+0000","last_undegraded":"2024-04-25T20:57:38.522987+0000","last_fullsized":"2024-04-25T20:57:38.522987+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:30:04.446592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"93'10","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.691387+0000","last_change":"2024-04-25T20:54:29.390334+0000","last_active":"2024-04-25T20:57:38.691387+0000","last_peered":"2024-04-25T20:57:38.691387+0000","last_clean":"2024-04-25T20:57:38.691387+0000","last_became_active":"2024-04-25T20:54:29.389943+0000","last_became_peered":"2024-04-25T20:54:29.389943+0000","last_unstale":"2024-04-25T20:57:38.691387+0000","last_undegraded":"2024-04-25T20:57:38.691387+0000","last_fullsized":"2024-04-25T20:57:38.691387+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:49:28.900512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"93'11","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:38.614712+0000","last_change":"2024-04-25T20:54:29.393917+0000","last_active":"2024-04-25T20:57:38.614712+0000","last_peered":"2024-04-25T20:57:38.614712+0000","last_clean":"2024-04-25T20:57:38.614712+0000","last_became_active":"2024-04-25T20:54:29.393607+0000","last_became_peered":"2024-04-25T20:54:29.393607+0000","last_unstale":"2024-04-25T20:57:38.614712+0000","last_undegraded":"2024-04-25T20:57:38.614712+0000","last_fullsized":"2024-04-25T20:57:38.614712+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:27:23.796935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"93'8","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-25T20:57:39.191048+0000","last_change":"2024-04-25T20:54:29.400635+0000","last_active":"2024-04-25T20:57:39.191048+0000","last_peered":"2024-04-25T20:57:39.191048+0000","last_clean":"2024-04-25T20:57:39.191048+0000","last_became_active":"2024-04-25T20:54:29.399913+0000","last_became_peered":"2024-04-25T20:54:29.399913+0000","last_unstale":"2024-04-25T20:57:39.191048+0000","last_undegraded":"2024-04-25T20:57:39.191048+0000","last_fullsized":"2024-04-25T20:57:39.191048+0000","mapping_epoch":88,"log_start":"0'0","ondisk_log_start":"0'0","created":88,"last_epoch_clean":89,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T20:54:28.362581+0000","last_clean_scrub_stamp":"2024-04-25T20:54:28.362581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T23:49:13.729591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797041,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1449,"num_read_kb":941,"num_write":428,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2617344,"data_stored":2391123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":314,"ondisk_log_size":314,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":83,"seq":356482285624,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32188,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712964,"statfs":{"total":95995035648,"available":95962075136,"internally_reserved":0,"allocated":1032192,"data_stored":504086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":75,"seq":322122547261,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37472,"kb_used_data":2016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93707680,"statfs":{"total":95995035648,"available":95956664320,"internally_reserved":0,"allocated":2064384,"data_stored":1494138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":69,"seq":296352743489,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36740,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708412,"statfs":{"total":95995035648,"available":95957413888,"internally_reserved":0,"allocated":1437696,"data_stored":897516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1599,"internal_metadata":36174273},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":61,"seq":261993005125,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36736,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708416,"statfs":{"total":95995035648,"available":95957417984,"internally_reserved":0,"allocated":1433600,"data_stored":897414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1638400,"data_stored":1097772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462993,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36348,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708804,"statfs":{"total":95995035648,"available":95957815296,"internally_reserved":0,"allocated":1032192,"data_stored":503689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":573452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1052672,"data_stored":504960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":576424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149791,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28868,"kb_used_data":1732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716284,"statfs":{"total":95995035648,"available":95965474816,"internally_reserved":0,"allocated":1773568,"data_stored":1225448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888154,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":504517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724630,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":29072,"kb_used_data":1936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716080,"statfs":{"total":95995035648,"available":95965265920,"internally_reserved":0,"allocated":1982464,"data_stored":1425297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":741376,"data_stored":721732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":360448,"data_stored":331330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":393932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":413696,"data_stored":394034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":400171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T20:57:50.057 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:57:50.296 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:51 smithi096 ceph-mon[29288]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-25T20:57:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:51 smithi097 ceph-mon[32667]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-25T20:57:51.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:51 smithi139 ceph-mon[33027]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 52 op/s 2024-04-25T20:57:51.978 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.975+0000 7f474affb700 1 -- 172.21.15.96:0/901848798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 msgr2=0x7f4744108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.975+0000 7f474affb700 1 --2- 172.21.15.96:0/901848798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f4744108e10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f4738009990 tx=0x7f4738009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.975+0000 7f474affb700 1 -- 172.21.15.96:0/901848798 shutdown_connections 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.975+0000 7f474affb700 1 --2- 172.21.15.96:0/901848798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f474406b0f0 0x7f474410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.975+0000 7f474affb700 1 --2- 172.21.15.96:0/901848798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f4744108e10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.975+0000 7f474affb700 1 --2- 172.21.15.96:0/901848798 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474406a750 0x7f474406abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- 172.21.15.96:0/901848798 >> 172.21.15.96:0/901848798 conn(0x7f47440788f0 msgr2=0x7f4744078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- 172.21.15.96:0/901848798 shutdown_connections 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- 172.21.15.96:0/901848798 wait complete. 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 Processor -- start 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- start start 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f474406a750 0x7f474419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474406b0f0 0x7f474419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:51.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f474419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4744112cb0 con 0x7f4744108a30 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4744112b30 con 0x7f474406a750 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f474affb700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4744112e30 con 0x7f474406b0f0 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f4749598700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f474419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f4749598700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f474419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33206/0 (socket says 172.21.15.96:33206) 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.976+0000 7f4749598700 1 -- 172.21.15.96:0/584834740 learned_addr learned my addr 172.21.15.96:0/584834740 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4748d97700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f474406a750 0x7f474419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4743fff700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474406b0f0 0x7f474419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4749598700 1 -- 172.21.15.96:0/584834740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474406b0f0 msgr2=0x7f474419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4749598700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474406b0f0 0x7f474419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4749598700 1 -- 172.21.15.96:0/584834740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f474406a750 msgr2=0x7f474419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4749598700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f474406a750 0x7f474419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:51.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4749598700 1 -- 172.21.15.96:0/584834740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47441a0450 con 0x7f4744108a30 2024-04-25T20:57:51.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4749598700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f474419fd10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b970 tx=0x7f473c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:51.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c019070 con 0x7f4744108a30 2024-04-25T20:57:51.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f473c00cde0 con 0x7f4744108a30 2024-04-25T20:57:51.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47441a0740 con 0x7f4744108a30 2024-04-25T20:57:51.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f473c007370 con 0x7f4744108a30 2024-04-25T20:57:51.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.977+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47441a7fc0 con 0x7f4744108a30 2024-04-25T20:57:51.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.978+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4744069730 con 0x7f4744108a30 2024-04-25T20:57:51.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.979+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f473c01c070 con 0x7f4744108a30 2024-04-25T20:57:51.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.980+0000 7f4741ffb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f47340a3ed0 0x7f47340a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:51.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.980+0000 7f4748d97700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f47340a3ed0 0x7f47340a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:51.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.980+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f473c0cfe80 con 0x7f4744108a30 2024-04-25T20:57:51.982 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.980+0000 7f4748d97700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f47340a3ed0 0x7f47340a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f47380098e0 tx=0x7f4738005960 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:51.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:51.984+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f473c09c450 con 0x7f4744108a30 2024-04-25T20:57:52.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.226+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f474419c7a0 con 0x7f4744108a30 2024-04-25T20:57:52.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.227+0000 7f4741ffb700 1 -- 172.21.15.96:0/584834740 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11859 (secure 0 0 0) 0x7f473c09bd50 con 0x7f4744108a30 2024-04-25T20:57:52.228 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:52.228 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:57:37.187405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T20:57:52.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f47340a3ed0 msgr2=0x7f47340a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f47340a3ed0 0x7f47340a6390 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f47380098e0 tx=0x7f4738005960 comp rx=0 tx=0).stop 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 msgr2=0x7f474419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f474419fd10 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b970 tx=0x7f473c00bc80 comp rx=0 tx=0).stop 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 shutdown_connections 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f47340a3ed0 0x7f47340a6390 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4744108a30 0x7f474419fd10 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f474406a750 0x7f474419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 --2- 172.21.15.96:0/584834740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f474406b0f0 0x7f474419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 >> 172.21.15.96:0/584834740 conn(0x7f47440788f0 msgr2=0x7f474410df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 shutdown_connections 2024-04-25T20:57:52.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:52.229+0000 7f474affb700 1 -- 172.21.15.96:0/584834740 wait complete. 2024-04-25T20:57:52.232 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T20:57:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:52 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/584834740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:52 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/584834740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:52.889 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:52 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/584834740' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:52.919 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:57:53.160 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:53.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:53 smithi097 ceph-mon[32667]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T20:57:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:53 smithi139 ceph-mon[33027]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T20:57:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:53 smithi096 ceph-mon[29288]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 -- 172.21.15.96:0/1175021141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c100d80 msgr2=0x7fcf2c101160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 --2- 172.21.15.96:0/1175021141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c100d80 0x7fcf2c101160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009900 tx=0x7fcf1c02a410 comp rx=0 tx=0).stop 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 -- 172.21.15.96:0/1175021141 shutdown_connections 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 --2- 172.21.15.96:0/1175021141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf2c10e930 0x7fcf2c110d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 --2- 172.21.15.96:0/1175021141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 0x7fcf2c10e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 --2- 172.21.15.96:0/1175021141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c100d80 0x7fcf2c101160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 -- 172.21.15.96:0/1175021141 >> 172.21.15.96:0/1175021141 conn(0x7fcf2c077fd0 msgr2=0x7fcf2c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 -- 172.21.15.96:0/1175021141 shutdown_connections 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.353+0000 7fcf33efc640 1 -- 172.21.15.96:0/1175021141 wait complete. 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 Processor -- start 2024-04-25T20:57:54.357 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 -- start start 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf2c100d80 0x7fcf2c1a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 0x7fcf2c1a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 0x7fcf2c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1143e0 con 0x7fcf2c10e930 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c114260 con 0x7fcf2c1016a0 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf33efc640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c114560 con 0x7fcf2c100d80 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf32499640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 0x7fcf2c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf32499640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 0x7fcf2c19a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33234/0 (socket says 172.21.15.96:33234) 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf32499640 1 -- 172.21.15.96:0/639195583 learned_addr learned my addr 172.21.15.96:0/639195583 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf31c98640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf2c100d80 0x7fcf2c1a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.354+0000 7fcf31497640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 0x7fcf2c1a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf32499640 1 -- 172.21.15.96:0/639195583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf2c100d80 msgr2=0x7fcf2c1a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf32499640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf2c100d80 0x7fcf2c1a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf32499640 1 -- 172.21.15.96:0/639195583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 msgr2=0x7fcf2c1a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf32499640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 0x7fcf2c1a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf32499640 1 -- 172.21.15.96:0/639195583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c19afe0 con 0x7fcf2c10e930 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf31497640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 0x7fcf2c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf32499640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 0x7fcf2c19a720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400eb40 tx=0x7fcf2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf22ffd640 1 -- 172.21.15.96:0/639195583 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24004b20 con 0x7fcf2c10e930 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c19b2d0 con 0x7fcf2c10e930 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf22ffd640 1 -- 172.21.15.96:0/639195583 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf24004cc0 con 0x7fcf2c10e930 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf22ffd640 1 -- 172.21.15.96:0/639195583 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf24010720 con 0x7fcf2c10e930 2024-04-25T20:57:54.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.355+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1a75e0 con 0x7fcf2c10e930 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.357+0000 7fcf22ffd640 1 -- 172.21.15.96:0/639195583 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7fcf24005560 con 0x7fcf2c10e930 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.358+0000 7fcf22ffd640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcf180a3ee0 0x7fcf180a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.358+0000 7fcf31c98640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcf180a3ee0 0x7fcf180a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.358+0000 7fcf22ffd640 1 -- 172.21.15.96:0/639195583 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcf240d10d0 con 0x7fcf2c10e930 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.358+0000 7fcf36c0b740 1 Processor -- start 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 -- start start 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e346fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e347cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9e33d4970 con 0x55d9e346ff40 2024-04-25T20:57:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9e33714a0 con 0x55d9e346f620 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9e33d0930 con 0x55d9e347a800 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31c98640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcf180a3ee0 0x7fcf180a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c031440 tx=0x7fcf1c005b50 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33248/0 (socket says 172.21.15.96:33248) 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 -- 172.21.15.96:0/615619232 learned_addr learned my addr 172.21.15.96:0/615619232 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31c98640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e346fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf32499640 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e347cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 -- 172.21.15.96:0/615619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 msgr2=0x55d9e347cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e347cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 -- 172.21.15.96:0/615619232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 msgr2=0x55d9e346fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e346fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.359+0000 7fcf31497640 1 -- 172.21.15.96:0/615619232 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e30e7210 con 0x55d9e346ff40 2024-04-25T20:57:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.360+0000 7fcf31497640 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347a2c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcf280114f0 tx=0x7fcf280119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf525e82742eeaa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.360+0000 7fcf17fff640 1 -- 172.21.15.96:0/615619232 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2801e070 con 0x55d9e346ff40 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.360+0000 7fcf17fff640 1 -- 172.21.15.96:0/615619232 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf280035f0 con 0x55d9e346ff40 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.360+0000 7fcf17fff640 1 -- 172.21.15.96:0/615619232 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2800f820 con 0x55d9e346ff40 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 -- 172.21.15.96:0/615619232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 msgr2=0x55d9e347a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347a2c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fcf280114f0 tx=0x7fcf280119b0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 -- 172.21.15.96:0/615619232 shutdown_connections 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e347cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347a2c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/615619232 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e346fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 -- 172.21.15.96:0/615619232 >> 172.21.15.96:0/615619232 conn(0x55d9e346bb30 msgr2=0x55d9e346bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 -- 172.21.15.96:0/615619232 shutdown_connections 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 -- 172.21.15.96:0/615619232 wait complete. 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 -- start start 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.361+0000 7fcf36c0b740 1 Processor -- start 2024-04-25T20:57:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e347a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34773a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34778e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d9e347d430 con 0x55d9e346ff40 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55d9e336f0b0 con 0x55d9e346f620 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55d9e347d5b0 con 0x55d9e347a800 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf36c0b740 10 client.0 _ll_register_callbacks cb 0x55d9e335cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34773a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34773a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33260/0 (socket says 172.21.15.96:33260) 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf31497640 1 -- 172.21.15.96:0/1579551849 learned_addr learned my addr 172.21.15.96:0/1579551849 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34778e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf31c98640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e347a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 -- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 msgr2=0x55d9e347a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e347a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 -- 172.21.15.96:0/1579551849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 msgr2=0x55d9e34773a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34773a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 -- 172.21.15.96:0/1579551849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e31b3620 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf31497640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf32499640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34778e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf240ce710 tx=0x7fcf2409c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b27a8bdfcd02d304 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf157fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.362+0000 7fcf157fa640 1 -- 172.21.15.96:0/1579551849 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf240bb060 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf280035f0 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf157fa640 1 -- 172.21.15.96:0/1579551849 <== mon.1 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcf240b62b0 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf157fa640 1 -- 172.21.15.96:0/1579551849 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf24082050 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 10 client.24832 fetch_fsmap learned FSMap version 26 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9e3489390 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf157fa640 1 -- 172.21.15.96:0/1579551849 <== mon.1 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf280035f0 con 0x55d9e347a800 2024-04-25T20:57:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf157fa640 1 -- 172.21.15.96:0/1579551849 <== mon.1 v2:172.21.15.139:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcf24098be0 con 0x55d9e347a800 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 10 client.24832 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 10 client.24832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 20 client.24832 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e348a800 0x55d9e348cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 4 client.24832 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9e33e2390 con 0x55d9e348a800 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.363+0000 7fcf31c98640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e348a800 0x55d9e348cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf31c98640 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e348a800 0x55d9e348cc10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94b1140b1873d03e server_cookie=701328f3b3802c17 in_seq=0 out_seq=0 2024-04-25T20:57:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf157fa640 10 client.24832 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf157fa640 1 -- 172.21.15.96:0/1579551849 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9e33e2390 con 0x55d9e348a800 2024-04-25T20:57:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf157fa640 10 client.24832 handle_command_reply: tid=0 2024-04-25T20:57:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf36c0b740 1 client.24832 shutdown 2024-04-25T20:57:54.371 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 msgr2=0x55d9e34778e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.364+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34778e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcf240ce710 tx=0x7fcf2409c8b0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 shutdown_connections 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e348a800 0x55d9e348cc10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34778e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34773a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/1579551849 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e347a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf157fa640 0 client.24832 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 >> 172.21.15.96:0/1579551849 conn(0x55d9e346bb30 msgr2=0x55d9e34718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 shutdown_connections 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 1 -- 172.21.15.96:0/1579551849 wait complete. 2024-04-25T20:57:54.372 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.365+0000 7fcf36c0b740 20 client.24832 trim_cache size 0 max 16384 2024-04-25T20:57:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 Processor -- start 2024-04-25T20:57:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 -- start start 2024-04-25T20:57:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 0x55d9e348cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e347a800 0x55d9e34924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1b24e0 con 0x55d9e346ff40 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1afdb0 con 0x55d9e347a800 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.381+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1adc40 con 0x55d9e346f620 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 0x55d9e348cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34862f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33262/0 (socket says 172.21.15.96:33262) 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 0x55d9e348cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:54454/0 (socket says 172.21.15.96:54454) 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31497640 1 -- 172.21.15.96:0/3265441100 learned_addr learned my addr 172.21.15.96:0/3265441100 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf32499640 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e347a800 0x55d9e34924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 -- 172.21.15.96:0/3265441100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e347a800 msgr2=0x55d9e34924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e347a800 0x55d9e34924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 -- 172.21.15.96:0/3265441100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 msgr2=0x55d9e34862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34862f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 -- 172.21.15.96:0/3265441100 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9e346fd70 con 0x55d9e346f620 2024-04-25T20:57:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31497640 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf31c98640 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 0x55d9e348cbf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009230 tx=0x7fcf1c040000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6451b7275059956b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf177fe640 1 -- 172.21.15.96:0/3265441100 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c035040 con 0x55d9e346f620 2024-04-25T20:57:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf177fe640 1 -- 172.21.15.96:0/3265441100 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1c053bc0 con 0x55d9e346f620 2024-04-25T20:57:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.382+0000 7fcf177fe640 1 -- 172.21.15.96:0/3265441100 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c053910 con 0x55d9e346f620 2024-04-25T20:57:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 -- 172.21.15.96:0/3265441100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 msgr2=0x55d9e348cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 0x55d9e348cbf0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009230 tx=0x7fcf1c040000 comp rx=0 tx=0).stop 2024-04-25T20:57:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 -- 172.21.15.96:0/3265441100 shutdown_connections 2024-04-25T20:57:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e347a800 0x55d9e34924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e34862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/3265441100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e346f620 0x55d9e348cbf0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 -- 172.21.15.96:0/3265441100 >> 172.21.15.96:0/3265441100 conn(0x55d9e346bb30 msgr2=0x55d9e346a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 -- 172.21.15.96:0/3265441100 shutdown_connections 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 -- 172.21.15.96:0/3265441100 wait complete. 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.383+0000 7fcf36c0b740 1 -- start start 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 Processor -- start 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e3482a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34802e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1ae290 con 0x55d9e346ff40 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1af6e0 con 0x55d9e346f620 2024-04-25T20:57:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c1ae940 con 0x55d9e347a800 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf36c0b740 10 client.0 _ll_register_callbacks cb 0x7fcf2409c9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf31497640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33274/0 (socket says 172.21.15.96:33274) 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf31497640 1 -- 172.21.15.96:0/2068441380 learned_addr learned my addr 172.21.15.96:0/2068441380 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf31c98640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e3482a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.384+0000 7fcf32499640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34802e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31497640 1 -- 172.21.15.96:0/2068441380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 msgr2=0x55d9e34802e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31497640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34802e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31497640 1 -- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 msgr2=0x55d9e3482a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31497640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e3482a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31497640 1 -- 172.21.15.96:0/2068441380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9e3480920 con 0x55d9e346ff40 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31c98640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e3482a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:57:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf31497640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347fda0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009250 tx=0x7fcf28009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eefcb56a65fb421 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf157fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf157fa640 1 -- 172.21.15.96:0/2068441380 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28027020 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf157fa640 1 -- 172.21.15.96:0/2068441380 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fcf28005250 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf1c053bc0 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf157fa640 1 -- 172.21.15.96:0/2068441380 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fcf28032050 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf36c0b740 10 client.15291 fetch_fsmap learned FSMap version 26 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d9e34f0bb0 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.385+0000 7fcf157fa640 1 -- 172.21.15.96:0/2068441380 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1c053bc0 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf157fa640 1 -- 172.21.15.96:0/2068441380 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fcf28023470 con 0x55d9e346ff40 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf36c0b740 10 client.15291 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf36c0b740 10 client.15291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf36c0b740 20 client.15291 populate_metadata read hostname 'smithi096' 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e34f1160 0x55d9e34f3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf36c0b740 4 client.15291 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:57:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9e33dc200 con 0x55d9e34f1160 2024-04-25T20:57:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.386+0000 7fcf31c98640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e34f1160 0x55d9e34f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.387+0000 7fcf157fa640 10 client.15291 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.387+0000 7fcf31c98640 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e34f1160 0x55d9e34f3580 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35be19c25022b426 server_cookie=3c4c31f9eb9a216a in_seq=0 out_seq=0 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf157fa640 1 -- 172.21.15.96:0/2068441380 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9e33dc200 con 0x55d9e34f1160 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf157fa640 10 client.15291 handle_command_reply: tid=0 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 client.15291 shutdown 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 msgr2=0x55d9e347fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347fda0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009250 tx=0x7fcf28009ce0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 shutdown_connections 2024-04-25T20:57:54.399 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55d9e34f1160 0x55d9e34f3580 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55d9e347a800 0x55d9e34802e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9e346ff40 0x55d9e347fda0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 --2- 172.21.15.96:0/2068441380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55d9e346f620 0x55d9e3482a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 >> 172.21.15.96:0/2068441380 conn(0x55d9e346bb30 msgr2=0x55d9e346acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.397+0000 7fcf157fa640 0 client.15291 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 shutdown_connections 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf36c0b740 1 -- 172.21.15.96:0/2068441380 wait complete. 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf36c0b740 20 client.15291 trim_cache size 0 max 16384 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T20:57:54.400 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T20:57:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T20:57:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcf180a3ee0 msgr2=0x7fcf180a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.401 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcf180a3ee0 0x7fcf180a63a0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c031440 tx=0x7fcf1c005b50 comp rx=0 tx=0).stop 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 msgr2=0x7fcf2c19a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 0x7fcf2c19a720 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcf2400eb40 tx=0x7fcf2400c2d0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 shutdown_connections 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcf180a3ee0 0x7fcf180a63a0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf2c10e930 0x7fcf2c19a720 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf2c1016a0 0x7fcf2c1a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 --2- 172.21.15.96:0/639195583 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcf2c100d80 0x7fcf2c1a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 >> 172.21.15.96:0/639195583 conn(0x7fcf2c077fd0 msgr2=0x7fcf2c10fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 shutdown_connections 2024-04-25T20:57:54.402 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:57:54.398+0000 7fcf33efc640 1 -- 172.21.15.96:0/639195583 wait complete. 2024-04-25T20:57:54.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 -- 172.21.15.96:0/1790785948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 msgr2=0x7f67d0074c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/1790785948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d0074c70 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009990 tx=0x7f67c0009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 -- 172.21.15.96:0/1790785948 shutdown_connections 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/1790785948 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67d0113740 0x7f67d0115b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/1790785948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d0074c70 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/1790785948 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67d00751b0 0x7f67d0075630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 -- 172.21.15.96:0/1790785948 >> 172.21.15.96:0/1790785948 conn(0x7f67d0079c00 msgr2=0x7f67d007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 -- 172.21.15.96:0/1790785948 shutdown_connections 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.832+0000 7f67d5cf2700 1 -- 172.21.15.96:0/1790785948 wait complete. 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 Processor -- start 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 -- start start 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67d00751b0 0x7f67d01a53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d01a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67d0113740 0x7f67d019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67d0119230 con 0x7f67d0076a20 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f67d01190b0 con 0x7f67d00751b0 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.833+0000 7f67d5cf2700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f67d01193b0 con 0x7f67d0113740 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d01a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d01a5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33284/0 (socket says 172.21.15.96:33284) 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 -- 172.21.15.96:0/689307814 learned_addr learned my addr 172.21.15.96:0/689307814 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67cf7fe700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67d00751b0 0x7f67d01a53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67cffff700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67d0113740 0x7f67d019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 -- 172.21.15.96:0/689307814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67d0113740 msgr2=0x7f67d019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67d0113740 0x7f67d019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 -- 172.21.15.96:0/689307814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67d00751b0 msgr2=0x7f67d01a53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67d00751b0 0x7f67d01a53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:54.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 -- 172.21.15.96:0/689307814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d019fcf0 con 0x7f67d0076a20 2024-04-25T20:57:54.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ceffd700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d01a5910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f67c400d980 tx=0x7f67c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4016070 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c4004830 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d019ffe0 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c4005540 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.834+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d01ac2a0 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.835+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d0069730 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.837+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f67c4004220 con 0x7f67d0076a20 2024-04-25T20:57:54.838 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.837+0000 7f67ccff9700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f67b80a3f50 0x7f67b80a6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.837+0000 7f67cf7fe700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f67b80a3f50 0x7f67b80a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.837+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f67c40d0c90 con 0x7f67d0076a20 2024-04-25T20:57:54.839 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.838+0000 7f67cf7fe700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f67b80a3f50 0x7f67b80a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67c00098e0 tx=0x7f67c0005960 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:54.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:54.841+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67c401f020 con 0x7f67d0076a20 2024-04-25T20:57:55.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.086+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67d01a0ac0 con 0x7f67d0076a20 2024-04-25T20:57:55.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.087+0000 7f67ccff9700 1 -- 172.21.15.96:0/689307814 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11859 (secure 0 0 0) 0x7f67d01a0ac0 con 0x7f67d0076a20 2024-04-25T20:57:55.088 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:55.088 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:57:37.187405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f67b80a3f50 msgr2=0x7f67b80a6410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f67b80a3f50 0x7f67b80a6410 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f67c00098e0 tx=0x7f67c0005960 comp rx=0 tx=0).stop 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 msgr2=0x7f67d01a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d01a5910 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f67c400d980 tx=0x7f67c400dc90 comp rx=0 tx=0).stop 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 shutdown_connections 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f67b80a3f50 0x7f67b80a6410 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f67d0113740 0x7f67d019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67d00751b0 0x7f67d01a53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:55.090 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 --2- 172.21.15.96:0/689307814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67d0076a20 0x7f67d01a5910 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:55.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 >> 172.21.15.96:0/689307814 conn(0x7f67d0079c00 msgr2=0x7f67d0111730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:55.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 shutdown_connections 2024-04-25T20:57:55.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:55.089+0000 7f67d5cf2700 1 -- 172.21.15.96:0/689307814 wait complete. 2024-04-25T20:57:55.091 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T20:57:55.754 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:55 smithi096 ceph-mon[29288]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-04-25T20:57:55.755 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:55 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/689307814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:55.756 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T20:57:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:55 smithi097 ceph-mon[32667]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-04-25T20:57:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:55 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/689307814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:55 smithi139 ceph-mon[33027]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 78 KiB/s wr, 50 op/s 2024-04-25T20:57:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:55 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/689307814' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:57:55.993 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 -- 172.21.15.96:0/3967406750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4107920 msgr2=0x7f3df4107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 --2- 172.21.15.96:0/3967406750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4107920 0x7f3df4107d00 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3de4009990 tx=0x7f3de4009ca0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 -- 172.21.15.96:0/3967406750 shutdown_connections 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 --2- 172.21.15.96:0/3967406750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3df4102330 0x7f3df410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 --2- 172.21.15.96:0/3967406750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4107920 0x7f3df4107d00 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 --2- 172.21.15.96:0/3967406750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df4101970 0x7f3df4101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 -- 172.21.15.96:0/3967406750 >> 172.21.15.96:0/3967406750 conn(0x7f3df4078910 msgr2=0x7f3df4078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 -- 172.21.15.96:0/3967406750 shutdown_connections 2024-04-25T20:57:57.667 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.666+0000 7f3dfa463700 1 -- 172.21.15.96:0/3967406750 wait complete. 2024-04-25T20:57:57.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 Processor -- start 2024-04-25T20:57:57.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 -- start start 2024-04-25T20:57:57.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 0x7f3df4199240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df4102330 0x7f3df4199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3df4107920 0x7f3df419dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4111b60 con 0x7f3df4101970 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3df41119e0 con 0x7f3df4102330 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3dfa463700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3df4111ce0 con 0x7f3df4107920 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3df3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 0x7f3df4199240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3df3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 0x7f3df4199240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45210/0 (socket says 172.21.15.96:45210) 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.667+0000 7f3df3fff700 1 -- 172.21.15.96:0/340423486 learned_addr learned my addr 172.21.15.96:0/340423486 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df8a00700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3df4107920 0x7f3df419dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df37fe700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df4102330 0x7f3df4199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df3fff700 1 -- 172.21.15.96:0/340423486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3df4107920 msgr2=0x7f3df419dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df3fff700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3df4107920 0x7f3df419dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df3fff700 1 -- 172.21.15.96:0/340423486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df4102330 msgr2=0x7f3df4199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.670 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df3fff700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df4102330 0x7f3df4199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df3fff700 1 -- 172.21.15.96:0/340423486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df419e200 con 0x7f3df4101970 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df3fff700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 0x7f3df4199240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4027070 con 0x7f3df4101970 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df419e490 con 0x7f3df4101970 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de4003f20 con 0x7f3df4101970 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4022cf0 con 0x7f3df4101970 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.668+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df4103420 con 0x7f3df4101970 2024-04-25T20:57:57.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.669+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3df4069730 con 0x7f3df4101970 2024-04-25T20:57:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.671+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138873+0+0 (secure 0 0 0) 0x7f3de402a070 con 0x7f3df4101970 2024-04-25T20:57:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.671+0000 7f3df17fa700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f3ddc0a3ed0 0x7f3ddc0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:57:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.671+0000 7f3df37fe700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f3ddc0a3ed0 0x7f3ddc0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:57:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.672+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f3de40d1bc0 con 0x7f3df4101970 2024-04-25T20:57:57.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.672+0000 7f3df37fe700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f3ddc0a3ed0 0x7f3ddc0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005950 tx=0x7f3de80058e0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:57:57.675 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.675+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3de4013070 con 0x7f3df4101970 2024-04-25T20:57:57.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:57 smithi096 ceph-mon[29288]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-04-25T20:57:57.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:57 smithi097 ceph-mon[32667]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-04-25T20:57:57.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:57 smithi139 ceph-mon[33027]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 79 KiB/s wr, 54 op/s 2024-04-25T20:57:57.875 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.875+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3df402df80 con 0x7f3df4101970 2024-04-25T20:57:57.877 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.877+0000 7f3df17fa700 1 -- 172.21.15.96:0/340423486 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+22896 (secure 0 0 0) 0x7f3de409e110 con 0x7f3df4101970 2024-04-25T20:57:57.877 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:57:57.877 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":94,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T20:57:38.516173+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2385969372":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/1528469114":"2024-04-26T20:57:38.516139+0000","172.21.15.96:6800/23858312":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/995634760":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6801/23858312":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/2934738303":"2024-04-26T20:57:38.516139+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T20:57:57.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f3ddc0a3ed0 msgr2=0x7f3ddc0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f3ddc0a3ed0 0x7f3ddc0a6390 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3de8005950 tx=0x7f3de80058e0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 msgr2=0x7f3df4199240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:57:57.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 0x7f3df4199240 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de40047b0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 shutdown_connections 2024-04-25T20:57:57.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f3ddc0a3ed0 0x7f3ddc0a6390 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3df4107920 0x7f3df419dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3df4101970 0x7f3df4199240 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 --2- 172.21.15.96:0/340423486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3df4102330 0x7f3df4199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:57:57.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 >> 172.21.15.96:0/340423486 conn(0x7f3df4078910 msgr2=0x7f3df406b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:57:57.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 shutdown_connections 2024-04-25T20:57:57.880 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:57:57.879+0000 7f3dfa463700 1 -- 172.21.15.96:0/340423486 wait complete. 2024-04-25T20:57:58.635 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-25T20:57:58.872 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:57:58.898 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:58 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/340423486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:58.994 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:58 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/340423486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:59.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:58 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/340423486' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T20:57:59.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:57:59 smithi139 ceph-mon[33027]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:57:59.942 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:57:59 smithi096 ceph-mon[29288]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:58:00.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:57:59 smithi097 ceph-mon[32667]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:58:00.521 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:58:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:58:00.410+0000 7f850f7e6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:58:00.522 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:58:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:58:00.410+0000 7f850f7e6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:58:00.522 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:58:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:58:00.410+0000 7f850f7e6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:58:00.522 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:58:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:58:00.410+0000 7f850f7e6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:58:00.522 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 20:58:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T20:58:00.410+0000 7f850f7e6700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-25T20:58:01.277 INFO:teuthology.orchestra.run.smithi096.stdout:{"run_dir":"/var/run/ceph"} 2024-04-25T20:58:01.277 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-25T20:58:01.278 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 2919 2024-04-25T20:58:01.278 DEBUG:tasks.quiescer:Yielding 2024-04-25T20:58:01.278 INFO:teuthology.run_tasks:Running task workunit... 2024-04-25T20:58:01.293 INFO:tasks.workunit:Pulling workunits from ref e221b808d285b9d02e54cc10c53c84188e8e41e3 2024-04-25T20:58:01.293 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-25T20:58:01.293 INFO:tasks.workunit:timeout=3h 2024-04-25T20:58:01.293 INFO:tasks.workunit:cleanup=True 2024-04-25T20:58:01.293 DEBUG:teuthology.orchestra.run.smithi096:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-25T20:58:01.294 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-04-25 20:55:02.658716025 +0000 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-04-25 20:55:02.658716025 +0000 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-04-25 20:57:36.477536236 +0000 2024-04-25T20:58:01.311 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-04-25 20:55:02.658716025 +0000 2024-04-25T20:58:01.311 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-25T20:58:01.312 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-25T20:58:01.383 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout e221b808d285b9d02e54cc10c53c84188e8e41e3 2024-04-25T20:58:01.439 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-25T20:58:01.550 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:01.821 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:01 smithi096 ceph-mon[29288]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T20:58:01.822 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:01 smithi096 ceph-mon[29288]: mgrmap e25: x(active, since 22s), standbys: y, z 2024-04-25T20:58:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:01 smithi097 ceph-mon[32667]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T20:58:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:01 smithi097 ceph-mon[32667]: mgrmap e25: x(active, since 22s), standbys: y, z 2024-04-25T20:58:02.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:01 smithi139 ceph-mon[33027]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-04-25T20:58:02.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:01 smithi139 ceph-mon[33027]: mgrmap e25: x(active, since 22s), standbys: y, z 2024-04-25T20:58:03.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 -- 172.21.15.96:0/3051006755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624102af0 msgr2=0x7f5624102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 --2- 172.21.15.96:0/3051006755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624102af0 0x7f5624102f70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5614009990 tx=0x7f5614009ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 -- 172.21.15.96:0/3051006755 shutdown_connections 2024-04-25T20:58:03.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 --2- 172.21.15.96:0/3051006755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56241034b0 0x7f562410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 --2- 172.21.15.96:0/3051006755 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f5624108a80 0x7f5624108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 --2- 172.21.15.96:0/3051006755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624102af0 0x7f5624102f70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 -- 172.21.15.96:0/3051006755 >> 172.21.15.96:0/3051006755 conn(0x7f5624079ad0 msgr2=0x7f5624079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 -- 172.21.15.96:0/3051006755 shutdown_connections 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.315+0000 7f562b796700 1 -- 172.21.15.96:0/3051006755 wait complete. 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.316+0000 7f562b796700 1 Processor -- start 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.316+0000 7f562b796700 1 -- start start 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f562b796700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5624102af0 0x7f5624199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f562b796700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56241034b0 0x7f56241997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f562b796700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 0x7f562419db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f562b796700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112c90 con 0x7f5624108a80 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f562b796700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112b10 con 0x7f5624102af0 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f562b796700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f5624112e10 con 0x7f56241034b0 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 0x7f562419db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 0x7f562419db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45228/0 (socket says 172.21.15.96:45228) 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 -- 172.21.15.96:0/1234684097 learned_addr learned my addr 172.21.15.96:0/1234684097 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5628d31700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56241034b0 0x7f56241997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629532700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5624102af0 0x7f5624199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 -- 172.21.15.96:0/1234684097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56241034b0 msgr2=0x7f56241997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56241034b0 0x7f56241997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 -- 172.21.15.96:0/1234684097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5624102af0 msgr2=0x7f5624199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5624102af0 0x7f5624199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 -- 172.21.15.96:0/1234684097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f562419e270 con 0x7f5624108a80 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629532700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5624102af0 0x7f5624199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5628d31700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56241034b0 0x7f56241997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f5629d33700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 0x7f562419db30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f561c007ee0 tx=0x7f561c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.317+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c00dbd0 con 0x7f5624108a80 2024-04-25T20:58:03.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.318+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f562419e560 con 0x7f5624108a80 2024-04-25T20:58:03.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.318+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f561c00dd70 con 0x7f5624108a80 2024-04-25T20:58:03.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.318+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c005990 con 0x7f5624108a80 2024-04-25T20:58:03.322 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.318+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a5de0 con 0x7f5624108a80 2024-04-25T20:58:03.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.319+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5624069730 con 0x7f5624108a80 2024-04-25T20:58:03.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.320+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f561c01a020 con 0x7f5624108a80 2024-04-25T20:58:03.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.320+0000 7f561a7fc700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f56100a3f60 0x7f56100a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:03.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.321+0000 7f5629532700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f56100a3f60 0x7f56100a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:03.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.321+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f561c0d0970 con 0x7f5624108a80 2024-04-25T20:58:03.323 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.321+0000 7f5629532700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f56100a3f60 0x7f56100a6420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56200099d0 tx=0x7f5620008040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:03.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.329+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f561c010070 con 0x7f5624108a80 2024-04-25T20:58:03.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.611+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f562419a5c0 con 0x7f5624108a80 2024-04-25T20:58:03.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.612+0000 7f561a7fc700 1 -- 172.21.15.96:0/1234684097 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11859 (secure 0 0 0) 0x7f561c09ce40 con 0x7f5624108a80 2024-04-25T20:58:03.613 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:58:03.613 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:57:37.187405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T20:58:03.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f56100a3f60 msgr2=0x7f56100a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f56100a3f60 0x7f56100a6420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56200099d0 tx=0x7f5620008040 comp rx=0 tx=0).stop 2024-04-25T20:58:03.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 msgr2=0x7f562419db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:03.615 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 0x7f562419db30 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f561c007ee0 tx=0x7f561c00d3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 shutdown_connections 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f56100a3f60 0x7f56100a6420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5624108a80 0x7f562419db30 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5624102af0 0x7f5624199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 --2- 172.21.15.96:0/1234684097 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f56241034b0 0x7f56241997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 >> 172.21.15.96:0/1234684097 conn(0x7f5624079ad0 msgr2=0x7f56241018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 shutdown_connections 2024-04-25T20:58:03.616 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:03.614+0000 7f562b796700 1 -- 172.21.15.96:0/1234684097 wait complete. 2024-04-25T20:58:03.617 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T20:58:03.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:03 smithi097 ceph-mon[32667]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:58:03.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:03 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1234684097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:03.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:03 smithi096 ceph-mon[29288]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:58:03.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:03 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1234684097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:04.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:03 smithi139 ceph-mon[33027]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-04-25T20:58:04.123 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:03 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1234684097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:04.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T20:54:31.232491+0000', 'modified': '2024-04-25T20:57:37.187405+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24443, 'mds_1': 24442, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24398': {'gid': 24398, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6835/122204709', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 122204709}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 122204709}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6835/1172572610', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6834', 'nonce': 1172572610}, {'type': 'v1', 'addr': '172.21.15.139:6835', 'nonce': 1172572610}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.139:6837/57160731', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6836', 'nonce': 57160731}, {'type': 'v1', 'addr': '172.21.15.139:6837', 'nonce': 57160731}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.97:6837/16128615', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 16128615}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 16128615}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.139:6839/2573809872', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.139:6838', 'nonce': 2573809872}, {'type': 'v1', 'addr': '172.21.15.139:6839', 'nonce': 2573809872}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.97:6839/3461334961', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3461334961}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3461334961}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24443, 'qdb_cluster': [24442, 24443, 24422]} 2024-04-25T20:58:04.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-04-25T20:58:04.300 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-25T20:58:04.602 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:05.747 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:05 smithi096 ceph-mon[29288]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-25T20:58:06.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:05 smithi097 ceph-mon[32667]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-25T20:58:06.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:05 smithi139 ceph-mon[33027]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 11 op/s 2024-04-25T20:58:06.306 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 -- 172.21.15.96:0/3532979271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 msgr2=0x7f4560105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 --2- 172.21.15.96:0/3532979271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f4560105d10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f454c0099c0 tx=0x7f454c009cd0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 -- 172.21.15.96:0/3532979271 shutdown_connections 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 --2- 172.21.15.96:0/3532979271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f45601063f0 0x7f456010dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 --2- 172.21.15.96:0/3532979271 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4560101360 0x7f4560101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 --2- 172.21.15.96:0/3532979271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f4560105d10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 -- 172.21.15.96:0/3532979271 >> 172.21.15.96:0/3532979271 conn(0x7f4560077fb0 msgr2=0x7f45600ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.303+0000 7f4566f2e700 1 -- 172.21.15.96:0/3532979271 shutdown_connections 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 -- 172.21.15.96:0/3532979271 wait complete. 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 Processor -- start 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 -- start start 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560101360 0x7f4560199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f456019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45601063f0 0x7f45601a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.307 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4560111260 con 0x7f4560101d10 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f45601110e0 con 0x7f4560101360 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.304+0000 7f4566f2e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f45601113e0 con 0x7f45601063f0 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f456019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f456019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45254/0 (socket says 172.21.15.96:45254) 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 -- 172.21.15.96:0/2277771629 learned_addr learned my addr 172.21.15.96:0/2277771629 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f4564cca700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560101360 0x7f4560199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f45654cb700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45601063f0 0x7f45601a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 -- 172.21.15.96:0/2277771629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45601063f0 msgr2=0x7f45601a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45601063f0 0x7f45601a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 -- 172.21.15.96:0/2277771629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560101360 msgr2=0x7f4560199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560101360 0x7f4560199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 -- 172.21.15.96:0/2277771629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45601a54e0 con 0x7f4560101d10 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455ffff700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f456019a460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f454c009f50 tx=0x7f454c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455dffb700 1 -- 172.21.15.96:0/2277771629 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c027070 con 0x7f4560101d10 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455dffb700 1 -- 172.21.15.96:0/2277771629 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c003fa0 con 0x7f4560101d10 2024-04-25T20:58:06.308 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f455dffb700 1 -- 172.21.15.96:0/2277771629 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c022d20 con 0x7f4560101d10 2024-04-25T20:58:06.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.305+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45601a56b0 con 0x7f4560101d10 2024-04-25T20:58:06.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.306+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45601a5a50 con 0x7f4560101d10 2024-04-25T20:58:06.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.308+0000 7f455dffb700 1 -- 172.21.15.96:0/2277771629 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f454c02a070 con 0x7f4560101d10 2024-04-25T20:58:06.309 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.309+0000 7f455dffb700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f45500a3fe0 0x7f45500a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.309+0000 7f4564cca700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f45500a3fe0 0x7f45500a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.309+0000 7f455dffb700 1 -- 172.21.15.96:0/2277771629 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f454c0d18b0 con 0x7f4560101d10 2024-04-25T20:58:06.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.309+0000 7f4564cca700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f45500a3fe0 0x7f45500a64a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f455400ac50 tx=0x7f455400a380 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:06.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.309+0000 7f45717b7b80 1 Processor -- start 2024-04-25T20:58:06.310 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.309+0000 7f45717b7b80 1 -- start start 2024-04-25T20:58:06.311 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3cdf7d0 0x5632d3cdfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 0x5632d3ce4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45717b7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632d3c31ce0 con 0x5632d3ce00f0 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45717b7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632d3bd8910 con 0x5632d3ce4580 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45717b7b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5632d3c2dca0 con 0x5632d3cdf7d0 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 0x5632d3ce4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 0x5632d3ce4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45264/0 (socket says 172.21.15.96:45264) 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 -- 172.21.15.96:0/2030875750 learned_addr learned my addr 172.21.15.96:0/2030875750 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f4564cca700 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3cdf7d0 0x5632d3cdfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f45654cb700 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 -- 172.21.15.96:0/2030875750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3cdf7d0 msgr2=0x5632d3cdfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3cdf7d0 0x5632d3cdfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.312 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 -- 172.21.15.96:0/2030875750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 msgr2=0x5632d3ce49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 -- 172.21.15.96:0/2030875750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d3b7f880 con 0x5632d3ce00f0 2024-04-25T20:58:06.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f455ffff700 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 0x5632d3ce4040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f454c09d6a0 tx=0x7f454c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=904e986d6eba77b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:06.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f454affd700 1 -- 172.21.15.96:0/2030875750 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c0880e0 con 0x5632d3ce00f0 2024-04-25T20:58:06.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f454affd700 1 -- 172.21.15.96:0/2030875750 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c09de10 con 0x5632d3ce00f0 2024-04-25T20:58:06.313 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.310+0000 7f454affd700 1 -- 172.21.15.96:0/2030875750 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c09dbd0 con 0x5632d3ce00f0 2024-04-25T20:58:06.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 -- 172.21.15.96:0/2030875750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 msgr2=0x5632d3ce4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.316 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 0x5632d3ce4040 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f454c09d6a0 tx=0x7f454c009310 comp rx=0 tx=0).stop 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 -- 172.21.15.96:0/2030875750 shutdown_connections 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3cdf7d0 0x5632d3cdfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2030875750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce00f0 0x5632d3ce4040 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 -- 172.21.15.96:0/2030875750 >> 172.21.15.96:0/2030875750 conn(0x5632d3c0c640 msgr2=0x5632d3cda820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 -- 172.21.15.96:0/2030875750 shutdown_connections 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.311+0000 7f45717b7b80 1 -- 172.21.15.96:0/2030875750 wait complete. 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 -- start start 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 Processor -- start 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ceb880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3cebdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.317 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632d3c31ce0 con 0x5632d3cdf7d0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632d3bd0b80 con 0x5632d3ce4580 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5632d3cec460 con 0x5632d3ce00f0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f45717b7b80 10 client.0 _ll_register_callbacks cb 0x5632d3bca900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f4564cca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ceb880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f4564cca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ceb880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45274/0 (socket says 172.21.15.96:45274) 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f4564cca700 1 -- 172.21.15.96:0/4283000700 learned_addr learned my addr 172.21.15.96:0/4283000700 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.312+0000 7f455ffff700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3cebdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f45654cb700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f4564cca700 1 -- 172.21.15.96:0/4283000700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 msgr2=0x5632d3cebdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f4564cca700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3cebdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f4564cca700 1 -- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 msgr2=0x5632d3ce8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f4564cca700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f4564cca700 1 -- 172.21.15.96:0/4283000700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d3be9220 con 0x5632d3cdf7d0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f4564cca700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ceb880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4554004660 tx=0x7f45540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83093b1919eb322e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:06.318 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f453ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f453ffff700 1 -- 172.21.15.96:0/4283000700 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554016070 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f453ffff700 1 -- 172.21.15.96:0/4283000700 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f4554010840 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c09de10 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f453ffff700 1 -- 172.21.15.96:0/4283000700 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4554003e20 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f45717b7b80 10 client.15327 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.313+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5632d3cff440 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f453ffff700 1 -- 172.21.15.96:0/4283000700 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c09de10 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f453ffff700 1 -- 172.21.15.96:0/4283000700 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f455402f050 con 0x5632d3cdf7d0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f45717b7b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f45717b7b80 10 client.15327 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f45717b7b80 20 client.15327 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f45717b7b80 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d01a20 0x5632d3d02e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f45717b7b80 4 client.15327 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632d3c3f4d0 con 0x5632d3d01a20 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.314+0000 7f455ffff700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d01a20 0x5632d3d02e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.319 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.315+0000 7f455ffff700 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d01a20 0x5632d3d02e90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=26633f6a62243d50 server_cookie=83b37ad3e9be0e45 in_seq=0 out_seq=0 2024-04-25T20:58:06.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.315+0000 7f453ffff700 10 client.15327 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:58:06.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.315+0000 7f453ffff700 1 -- 172.21.15.96:0/4283000700 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5632d3c3f4d0 con 0x5632d3d01a20 2024-04-25T20:58:06.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.315+0000 7f453ffff700 10 client.15327 handle_command_reply: tid=0 2024-04-25T20:58:06.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 client.15327 shutdown 2024-04-25T20:58:06.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 msgr2=0x5632d3ceb880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.320 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ceb880 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f4554004660 tx=0x7f45540047c0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 shutdown_connections 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d01a20 0x5632d3d02e90 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce4580 0x5632d3ce8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ceb880 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 --2- 172.21.15.96:0/4283000700 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3cebdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 >> 172.21.15.96:0/4283000700 conn(0x5632d3c0c640 msgr2=0x5632d3ce59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f453ffff700 0 client.15327 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 shutdown_connections 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.316+0000 7f45717b7b80 1 -- 172.21.15.96:0/4283000700 wait complete. 2024-04-25T20:58:06.321 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.317+0000 7f45717b7b80 20 client.15327 trim_cache size 0 max 16384 2024-04-25T20:58:06.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 Processor -- start 2024-04-25T20:58:06.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 -- start start 2024-04-25T20:58:06.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3d0c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 0x5632d3d0c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.341 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632d3d011d0 con 0x5632d3ce4580 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632d3d018a0 con 0x5632d3cdf7d0 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.339+0000 7f45717b7b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5632d3d01f70 con 0x5632d3ce00f0 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 0x5632d3d0c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f455ffff700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3d0c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 0x5632d3d0c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45278/0 (socket says 172.21.15.96:45278) 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 -- 172.21.15.96:0/1108634025 learned_addr learned my addr 172.21.15.96:0/1108634025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f4564cca700 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 -- 172.21.15.96:0/1108634025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 msgr2=0x5632d3d0c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3d0c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 -- 172.21.15.96:0/1108634025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3cdf7d0 msgr2=0x5632d3ce8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 -- 172.21.15.96:0/1108634025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632d3ce7340 con 0x5632d3ce4580 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f4564cca700 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f455ffff700 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3d0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:06.342 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f45654cb700 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 0x5632d3d0c830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45580135f0 tx=0x7f4558007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd21613ec09ff57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:06.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f454a7fc700 1 -- 172.21.15.96:0/1108634025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455801a030 con 0x5632d3ce4580 2024-04-25T20:58:06.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f454a7fc700 1 -- 172.21.15.96:0/1108634025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558004020 con 0x5632d3ce4580 2024-04-25T20:58:06.343 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.340+0000 7f454a7fc700 1 -- 172.21.15.96:0/1108634025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558016450 con 0x5632d3ce4580 2024-04-25T20:58:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 -- 172.21.15.96:0/1108634025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 msgr2=0x5632d3d0c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 0x5632d3d0c830 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45580135f0 tx=0x7f4558007950 comp rx=0 tx=0).stop 2024-04-25T20:58:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 -- 172.21.15.96:0/1108634025 shutdown_connections 2024-04-25T20:58:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3ce4580 0x5632d3d0c830 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.347 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 --2- 172.21.15.96:0/1108634025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce00f0 0x5632d3d0c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 -- 172.21.15.96:0/1108634025 >> 172.21.15.96:0/1108634025 conn(0x5632d3c0c640 msgr2=0x5632d3ce8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.341+0000 7f45717b7b80 1 -- 172.21.15.96:0/1108634025 shutdown_connections 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 -- 172.21.15.96:0/1108634025 wait complete. 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 -- start start 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 Processor -- start 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce00f0 0x5632d3ce1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce4580 0x5632d3ce1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5632d3d02d10 con 0x5632d3cdf7d0 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5632d3d02640 con 0x5632d3ce00f0 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5632d3d033e0 con 0x5632d3ce4580 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.342+0000 7f45717b7b80 10 client.0 _ll_register_callbacks cb 0x5632d3bca900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45282/0 (socket says 172.21.15.96:45282) 2024-04-25T20:58:06.348 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 -- 172.21.15.96:0/2936177664 learned_addr learned my addr 172.21.15.96:0/2936177664 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f455ffff700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce00f0 0x5632d3ce1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f45654cb700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce4580 0x5632d3ce1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 -- 172.21.15.96:0/2936177664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce4580 msgr2=0x5632d3ce1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce4580 0x5632d3ce1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 -- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce00f0 msgr2=0x5632d3ce1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce00f0 0x5632d3ce1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 -- 172.21.15.96:0/2936177664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632d3cfe410 con 0x5632d3cdf7d0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f45654cb700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce4580 0x5632d3ce1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f4564cca700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce0ec0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f455401a010 tx=0x7f4554027720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c7af53c673c1f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f453ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f453ffff700 1 -- 172.21.15.96:0/2936177664 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4554016040 con 0x5632d3cdf7d0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f453ffff700 1 -- 172.21.15.96:0/2936177664 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f45540197a0 con 0x5632d3cdf7d0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.343+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4558004020 con 0x5632d3cdf7d0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f453ffff700 1 -- 172.21.15.96:0/2936177664 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4554040050 con 0x5632d3cdf7d0 2024-04-25T20:58:06.349 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 10 client.15333 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5632d3d68e10 con 0x5632d3cdf7d0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f453ffff700 1 -- 172.21.15.96:0/2936177664 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4558004020 con 0x5632d3cdf7d0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f453ffff700 1 -- 172.21.15.96:0/2936177664 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f455402f070 con 0x5632d3cdf7d0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 10 client.15333 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 10 client.15333 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 20 client.15333 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d6daf0 0x5632d3d6c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 4 client.15333 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "status"}) -- 0x5632d3c39570 con 0x5632d3d6daf0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.344+0000 7f455ffff700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d6daf0 0x5632d3d6c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f453ffff700 10 client.15333 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f455ffff700 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d6daf0 0x5632d3d6c800 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8342db2fa77efc6a server_cookie=4ce02cd06fbcfb39 in_seq=0 out_seq=0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f453ffff700 1 -- 172.21.15.96:0/2936177664 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5632d3c39570 con 0x5632d3d6daf0 2024-04-25T20:58:06.350 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f453ffff700 10 client.15333 handle_command_reply: tid=0 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 client.15333 shutdown 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 msgr2=0x5632d3ce0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce0ec0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f455401a010 tx=0x7f4554027720 comp rx=0 tx=0).stop 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 shutdown_connections 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x5632d3d6daf0 0x5632d3d6c800 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5632d3ce4580 0x5632d3ce1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5632d3cdf7d0 0x5632d3ce0ec0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 --2- 172.21.15.96:0/2936177664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5632d3ce00f0 0x5632d3ce1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f453ffff700 0 client.15333 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.345+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 >> 172.21.15.96:0/2936177664 conn(0x5632d3c0c640 msgr2=0x5632d3cde5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 shutdown_connections 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f45717b7b80 1 -- 172.21.15.96:0/2936177664 wait complete. 2024-04-25T20:58:06.351 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f45717b7b80 20 client.15333 trim_cache size 0 max 16384 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 2, 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24422, 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 202.02045207800001, 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 26, 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 94, 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T20:58:06.352 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 245.68250286599999 2024-04-25T20:58:06.353 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:58:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f45500a3fe0 msgr2=0x7f45500a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f45500a3fe0 0x7f45500a64a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f455400ac50 tx=0x7f455400a380 comp rx=0 tx=0).stop 2024-04-25T20:58:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 msgr2=0x7f456019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:06.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f456019a460 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f454c009f50 tx=0x7f454c0037c0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 shutdown_connections 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f45500a3fe0 0x7f45500a64a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f45601063f0 0x7f45601a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4560101360 0x7f4560199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 --2- 172.21.15.96:0/2277771629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4560101d10 0x7f456019a460 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.346+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 >> 172.21.15.96:0/2277771629 conn(0x7f4560077fb0 msgr2=0x7f45600ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.347+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 shutdown_connections 2024-04-25T20:58:06.354 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:06.347+0000 7f4566f2e700 1 -- 172.21.15.96:0/2277771629 wait complete. 2024-04-25T20:58:07.093 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-25T20:58:07.357 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.486+0000 7f6402b76640 1 -- 172.21.15.96:0/1196876890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc106170 msgr2=0x7f63fc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.486+0000 7f6402b76640 1 --2- 172.21.15.96:0/1196876890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc106170 0x7f63fc10dd40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009ca0 tx=0x7f63f402a8c0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- 172.21.15.96:0/1196876890 shutdown_connections 2024-04-25T20:58:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 --2- 172.21.15.96:0/1196876890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc106170 0x7f63fc10dd40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.489 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 --2- 172.21.15.96:0/1196876890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63fc101aa0 0x7f63fc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 --2- 172.21.15.96:0/1196876890 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63fc1010f0 0x7f63fc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- 172.21.15.96:0/1196876890 >> 172.21.15.96:0/1196876890 conn(0x7f63fc077fd0 msgr2=0x7f63fc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- 172.21.15.96:0/1196876890 shutdown_connections 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- 172.21.15.96:0/1196876890 wait complete. 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 Processor -- start 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- start start 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 0x7f63fc102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63fc106170 0x7f63fc103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63fc1035b0 0x7f63fc1a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc111130 con 0x7f63fc1010f0 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc110fb0 con 0x7f63fc1035b0 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.487+0000 7f6402b76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1112b0 con 0x7f63fc106170 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 0x7f63fc102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 0x7f63fc102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45638/0 (socket says 172.21.15.96:45638) 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 -- 172.21.15.96:0/3307524510 learned_addr learned my addr 172.21.15.96:0/3307524510 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f63f3fff640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63fc106170 0x7f63fc103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6401113640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63fc1035b0 0x7f63fc1a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 -- 172.21.15.96:0/3307524510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63fc106170 msgr2=0x7f63fc103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63fc106170 0x7f63fc103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 -- 172.21.15.96:0/3307524510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63fc1035b0 msgr2=0x7f63fc1a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63fc1035b0 0x7f63fc1a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 -- 172.21.15.96:0/3307524510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63fc1a6f70 con 0x7f63fc1010f0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6400912640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 0x7f63fc102b30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b4a0 tx=0x7f63f800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f63f1ffb640 1 -- 172.21.15.96:0/3307524510 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8014070 con 0x7f63fc1010f0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63fc1a7260 con 0x7f63fc1010f0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f63f1ffb640 1 -- 172.21.15.96:0/3307524510 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f8010ac0 con 0x7f63fc1010f0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.488+0000 7f63f1ffb640 1 -- 172.21.15.96:0/3307524510 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8017c30 con 0x7f63fc1010f0 2024-04-25T20:58:07.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.489+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63fc1a7760 con 0x7f63fc1010f0 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f63f1ffb640 1 -- 172.21.15.96:0/3307524510 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f63f8017070 con 0x7f63fc1010f0 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 Processor -- start 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 -- start start 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2e8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f63f1ffb640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f63e40a3f70 0x7f63e40a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5639df24d970 con 0x5639df2e8f40 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5639df1ea4a0 con 0x5639df2f3800 2024-04-25T20:58:07.494 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f640596f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5639df249930 con 0x5639df2e8620 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45646/0 (socket says 172.21.15.96:45646) 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f63f3fff640 1 -- 172.21.15.96:0/135011188 learned_addr learned my addr 172.21.15.96:0/135011188 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f6400912640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f63e40a3f70 0x7f63e40a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.491+0000 7f63f1ffb640 1 -- 172.21.15.96:0/3307524510 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f63f80cfd10 con 0x7f63fc1010f0 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f6401113640 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f6401113640 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2e8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63f3fff640 1 -- 172.21.15.96:0/135011188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 msgr2=0x5639df2e8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63f3fff640 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2e8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63f3fff640 1 -- 172.21.15.96:0/135011188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 msgr2=0x5639df2f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63f3fff640 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63f3fff640 1 -- 172.21.15.96:0/135011188 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639def60210 con 0x5639df2e8f40 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f6401113640 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:07.495 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63f3fff640 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f32c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00abb0 tx=0x7f63ec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4843e56e2208ec85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f6400912640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f63e40a3f70 0x7f63e40a6430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63f80cd400 tx=0x7f63f8010f90 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63e2ffd640 1 -- 172.21.15.96:0/135011188 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec01d070 con 0x5639df2e8f40 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63e2ffd640 1 -- 172.21.15.96:0/135011188 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec018af0 con 0x5639df2e8f40 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.492+0000 7f63e2ffd640 1 -- 172.21.15.96:0/135011188 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec010a30 con 0x5639df2e8f40 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 -- 172.21.15.96:0/135011188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 msgr2=0x5639df2f32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f32c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00abb0 tx=0x7f63ec00f790 comp rx=0 tx=0).stop 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 -- 172.21.15.96:0/135011188 shutdown_connections 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f32c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 --2- 172.21.15.96:0/135011188 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2e8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 -- 172.21.15.96:0/135011188 >> 172.21.15.96:0/135011188 conn(0x5639df2e4b30 msgr2=0x5639df2e4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 -- 172.21.15.96:0/135011188 shutdown_connections 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 -- 172.21.15.96:0/135011188 wait complete. 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 -- start start 2024-04-25T20:58:07.496 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.493+0000 7f640596f740 1 Processor -- start 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5639df2f6430 con 0x5639df2e8f40 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5639df1e80b0 con 0x5639df2f3800 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5639df2f65b0 con 0x5639df2e8620 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f640596f740 10 client.0 _ll_register_callbacks cb 0x5639df1d5d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45652/0 (socket says 172.21.15.96:45652) 2024-04-25T20:58:07.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 -- 172.21.15.96:0/1974475825 learned_addr learned my addr 172.21.15.96:0/1974475825 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f6401113640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f6401113640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 -- 172.21.15.96:0/1974475825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 msgr2=0x5639df2f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 -- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 msgr2=0x5639df2f0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 -- 172.21.15.96:0/1974475825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639df02c620 con 0x5639df2e8f40 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f6401113640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63f3fff640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f04b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63ec018970 tx=0x7f63ec00a740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ad826363e10289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.494+0000 7f63dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f63dbfff640 1 -- 172.21.15.96:0/1974475825 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec01d050 con 0x5639df2e8f40 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f63dbfff640 1 -- 172.21.15.96:0/1974475825 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f63ec005450 con 0x5639df2e8f40 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ec018af0 con 0x5639df2e8f40 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f63dbfff640 1 -- 172.21.15.96:0/1974475825 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63ec032050 con 0x5639df2e8f40 2024-04-25T20:58:07.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 10 client.15345 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639df3021b0 con 0x5639df2e8f40 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f63dbfff640 1 -- 172.21.15.96:0/1974475825 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec018af0 con 0x5639df2e8f40 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f63dbfff640 1 -- 172.21.15.96:0/1974475825 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f63ec0046c0 con 0x5639df2e8f40 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 10 client.15345 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 10 client.15345 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 20 client.15345 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df3035b0 0x5639df3059a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 4 client.15345 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.495+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639df25b390 con 0x5639df3035b0 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.496+0000 7f6401113640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df3035b0 0x5639df3059a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.496+0000 7f6401113640 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df3035b0 0x5639df3059a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9837659059361a server_cookie=23df89c0c21bfb3d in_seq=0 out_seq=0 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.496+0000 7f63dbfff640 10 client.15345 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f63dbfff640 1 -- 172.21.15.96:0/1974475825 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639df25b390 con 0x5639df3035b0 2024-04-25T20:58:07.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f63dbfff640 10 client.15345 handle_command_reply: tid=0 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 client.15345 shutdown 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 msgr2=0x5639df2f04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f04b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f63ec018970 tx=0x7f63ec00a740 comp rx=0 tx=0).stop 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 shutdown_connections 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df3035b0 0x5639df3059a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2f3800 0x5639df2f0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f63dbfff640 0 client.15345 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2f04b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 --2- 172.21.15.96:0/1974475825 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2e8620 0x5639df2f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 >> 172.21.15.96:0/1974475825 conn(0x5639df2e4b30 msgr2=0x5639df2ea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 shutdown_connections 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 1 -- 172.21.15.96:0/1974475825 wait complete. 2024-04-25T20:58:07.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.497+0000 7f640596f740 20 client.15345 trim_cache size 0 max 16384 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 Processor -- start 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 -- start start 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 0x5639df2e4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8f40 0x5639df2f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df30af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1ae090 con 0x5639df2e8620 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1ab960 con 0x5639df2e8f40 2024-04-25T20:58:07.518 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f640596f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1a97d0 con 0x5639df2f3800 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f6401113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 0x5639df2e4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f6401113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 0x5639df2e4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45668/0 (socket says 172.21.15.96:45668) 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f6401113640 1 -- 172.21.15.96:0/2871088008 learned_addr learned my addr 172.21.15.96:0/2871088008 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.516+0000 7f63f3fff640 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8f40 0x5639df2f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df30af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 -- 172.21.15.96:0/2871088008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 msgr2=0x5639df30af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df30af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 -- 172.21.15.96:0/2871088008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8f40 msgr2=0x5639df2f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8f40 0x5639df2f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 -- 172.21.15.96:0/2871088008 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639df2f5e20 con 0x5639df2e8620 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f6401113640 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 0x5639df2e4430 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009ca0 tx=0x7f63f4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39ee2c559fbb517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f63e27fc640 1 -- 172.21.15.96:0/2871088008 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4056bd0 con 0x5639df2e8620 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f63e27fc640 1 -- 172.21.15.96:0/2871088008 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f4036040 con 0x5639df2e8620 2024-04-25T20:58:07.519 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.517+0000 7f63e27fc640 1 -- 172.21.15.96:0/2871088008 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f404c960 con 0x5639df2e8620 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 -- 172.21.15.96:0/2871088008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 msgr2=0x5639df2e4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 0x5639df2e4430 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009ca0 tx=0x7f63f4009360 comp rx=0 tx=0).stop 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 -- 172.21.15.96:0/2871088008 shutdown_connections 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df30af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8f40 0x5639df2f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 --2- 172.21.15.96:0/2871088008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8620 0x5639df2e4430 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 -- 172.21.15.96:0/2871088008 >> 172.21.15.96:0/2871088008 conn(0x5639df2e4b30 msgr2=0x5639df2ea860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 -- 172.21.15.96:0/2871088008 shutdown_connections 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 -- 172.21.15.96:0/2871088008 wait complete. 2024-04-25T20:58:07.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 -- start start 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.518+0000 7f640596f740 1 Processor -- start 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8620 0x5639df2fb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2ecd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df2ed2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1a9e20 con 0x5639df2e8f40 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1ab290 con 0x5639df2e8620 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f63fc1aa4f0 con 0x5639df2f3800 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f640596f740 10 client.0 _ll_register_callbacks cb 0x7f63ec00aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2ecd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2ecd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45682/0 (socket says 172.21.15.96:45682) 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 -- 172.21.15.96:0/854396659 learned_addr learned my addr 172.21.15.96:0/854396659 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f6401113640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8620 0x5639df2fb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f6401113640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df2ed2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 -- 172.21.15.96:0/854396659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 msgr2=0x5639df2ed2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df2ed2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 -- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8620 msgr2=0x5639df2fb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8620 0x5639df2fb9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 -- 172.21.15.96:0/854396659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639df2ed890 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63f3fff640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2ecd60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5639df2f1a80 tx=0x7f63ec004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a2fbeb4a06532d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63dbfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63dbfff640 1 -- 172.21.15.96:0/854396659 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec01d040 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.519+0000 7f63dbfff640 1 -- 172.21.15.96:0/854396659 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f63ec005450 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f4036040 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f63dbfff640 1 -- 172.21.15.96:0/854396659 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f63ec004870 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 10 client.15351 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5639df3695f0 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f63dbfff640 1 -- 172.21.15.96:0/854396659 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f4036040 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f63dbfff640 1 -- 172.21.15.96:0/854396659 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f63ec046050 con 0x5639df2e8f40 2024-04-25T20:58:07.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 10 client.15351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 20 client.15351 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df369cb0 0x5639df36c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 4 client.15351 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639df255200 con 0x5639df369cb0 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.520+0000 7f6401113640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df369cb0 0x5639df36c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.521+0000 7f63dbfff640 10 client.15351 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:07.526 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.521+0000 7f6401113640 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df369cb0 0x5639df36c0d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1ce84e553e10f4 server_cookie=fd7768575eecc5ec in_seq=0 out_seq=0 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.529+0000 7f63dbfff640 1 -- 172.21.15.96:0/854396659 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639df255200 con 0x5639df369cb0 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.529+0000 7f63dbfff640 10 client.15351 handle_command_reply: tid=0 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.529+0000 7f640596f740 1 client.15351 shutdown 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 msgr2=0x5639df2ecd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2ecd60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x5639df2f1a80 tx=0x7f63ec004fb0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 shutdown_connections 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5639df369cb0 0x5639df36c0d0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5639df2f3800 0x5639df2ed2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f63dbfff640 0 client.15351 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5639df2e8f40 0x5639df2ecd60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 --2- 172.21.15.96:0/854396659 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5639df2e8620 0x5639df2fb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 >> 172.21.15.96:0/854396659 conn(0x5639df2e4b30 msgr2=0x5639df2eea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 shutdown_connections 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 1 -- 172.21.15.96:0/854396659 wait complete. 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f640596f740 20 client.15351 trim_cache size 0 max 16384 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T20:58:07.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T20:58:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f63e40a3f70 msgr2=0x7f63e40a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f6402b76640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f63e40a3f70 0x7f63e40a6430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63f80cd400 tx=0x7f63f8010f90 comp rx=0 tx=0).stop 2024-04-25T20:58:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 msgr2=0x7f63fc102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.530+0000 7f6402b76640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 0x7f63fc102b30 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63f800b4a0 tx=0x7f63f800b960 comp rx=0 tx=0).stop 2024-04-25T20:58:07.533 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 shutdown_connections 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f63e40a3f70 0x7f63e40a6430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f63fc1035b0 0x7f63fc1a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f63fc106170 0x7f63fc103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 --2- 172.21.15.96:0/3307524510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63fc1010f0 0x7f63fc102b30 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 >> 172.21.15.96:0/3307524510 conn(0x7f63fc077fd0 msgr2=0x7f63fc069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 shutdown_connections 2024-04-25T20:58:07.534 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:07.531+0000 7f6402b76640 1 -- 172.21.15.96:0/3307524510 wait complete. 2024-04-25T20:58:07.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:07 smithi096 ceph-mon[29288]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T20:58:08.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:07 smithi097 ceph-mon[32667]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T20:58:08.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:07 smithi139 ceph-mon[33027]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-04-25T20:58:09.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 -- 172.21.15.96:0/2510247560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e01019d0 msgr2=0x7f48e010f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/2510247560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e01019d0 0x7f48e010f910 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f48d800b0a0 tx=0x7f48d800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 -- 172.21.15.96:0/2510247560 shutdown_connections 2024-04-25T20:58:09.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/2510247560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e01019d0 0x7f48e010f910 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/2510247560 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e0108ad0 0x7f48e0108eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/2510247560 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48e0101030 0x7f48e0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 -- 172.21.15.96:0/2510247560 >> 172.21.15.96:0/2510247560 conn(0x7f48e0077fb0 msgr2=0x7f48e00ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 -- 172.21.15.96:0/2510247560 shutdown_connections 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 -- 172.21.15.96:0/2510247560 wait complete. 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 Processor -- start 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.213+0000 7f48e6b8e700 1 -- start start 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e0101030 0x7f48e0102210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48e01019d0 0x7f48e0102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 0x7f48e0102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0112c50 con 0x7f48e0108ad0 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0112ad0 con 0x7f48e0101030 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48e0112dd0 con 0x7f48e01019d0 2024-04-25T20:58:09.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 0x7f48e0102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 0x7f48e0102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45704/0 (socket says 172.21.15.96:45704) 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 -- 172.21.15.96:0/3976893526 learned_addr learned my addr 172.21.15.96:0/3976893526 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e492a700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e0101030 0x7f48e0102210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48dffff700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48e01019d0 0x7f48e0102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 -- 172.21.15.96:0/3976893526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48e01019d0 msgr2=0x7f48e0102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48e01019d0 0x7f48e0102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 -- 172.21.15.96:0/3976893526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e0101030 msgr2=0x7f48e0102210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e0101030 0x7f48e0102210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 -- 172.21.15.96:0/3976893526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48e01a7630 con 0x7f48e0108ad0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e512b700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 0x7f48e0102c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009fd0 tx=0x7f48d8004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48ddffb700 1 -- 172.21.15.96:0/3976893526 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d8021070 con 0x7f48e0108ad0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48ddffb700 1 -- 172.21.15.96:0/3976893526 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d8007b80 con 0x7f48e0108ad0 2024-04-25T20:58:09.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48e01a78c0 con 0x7f48e0108ad0 2024-04-25T20:58:09.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.214+0000 7f48ddffb700 1 -- 172.21.15.96:0/3976893526 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d80052a0 con 0x7f48e0108ad0 2024-04-25T20:58:09.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.215+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48e01a7dc0 con 0x7f48e0108ad0 2024-04-25T20:58:09.218 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.217+0000 7f48ddffb700 1 -- 172.21.15.96:0/3976893526 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f48d8024070 con 0x7f48e0108ad0 2024-04-25T20:58:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.218+0000 7f48ddffb700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48d00a3fe0 0x7f48d00a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.218+0000 7f48e492a700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48d00a3fe0 0x7f48d00a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.218+0000 7f48ddffb700 1 -- 172.21.15.96:0/3976893526 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f48d800f020 con 0x7f48e0108ad0 2024-04-25T20:58:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.218+0000 7f48e492a700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48d00a3fe0 0x7f48d00a64a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48d4005950 tx=0x7f48d400b410 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.218+0000 7f48f1417b80 1 Processor -- start 2024-04-25T20:58:09.219 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.218+0000 7f48f1417b80 1 -- start start 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d57d0 0x55563d4d5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4da040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 0x55563d4da9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48f1417b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55563d427ce0 con 0x55563d4da580 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48f1417b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55563d3ce910 con 0x55563d4d60f0 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48f1417b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55563d423ca0 con 0x55563d4d57d0 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 0x55563d4da9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 0x55563d4da9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45712/0 (socket says 172.21.15.96:45712) 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 -- 172.21.15.96:0/820327345 learned_addr learned my addr 172.21.15.96:0/820327345 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48dffff700 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4da040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e492a700 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d57d0 0x55563d4d5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 -- 172.21.15.96:0/820327345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d57d0 msgr2=0x55563d4d5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d57d0 0x55563d4d5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 -- 172.21.15.96:0/820327345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 msgr2=0x55563d4da040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4da040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 -- 172.21.15.96:0/820327345 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563d375880 con 0x55563d4da580 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48dffff700 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4da040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.219+0000 7f48e512b700 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 0x55563d4da9e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f48d809a3e0 tx=0x7f48d80099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701c3196d58bab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48caffd700 1 -- 172.21.15.96:0/820327345 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d809da10 con 0x55563d4da580 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48caffd700 1 -- 172.21.15.96:0/820327345 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d80a1dc0 con 0x55563d4da580 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48caffd700 1 -- 172.21.15.96:0/820327345 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d809a080 con 0x55563d4da580 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 -- 172.21.15.96:0/820327345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 msgr2=0x55563d4da9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 0x55563d4da9e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f48d809a3e0 tx=0x7f48d80099e0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 -- 172.21.15.96:0/820327345 shutdown_connections 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4da580 0x55563d4da9e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d57d0 0x55563d4d5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 --2- 172.21.15.96:0/820327345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4da040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 -- 172.21.15.96:0/820327345 >> 172.21.15.96:0/820327345 conn(0x55563d402640 msgr2=0x55563d4d0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.220+0000 7f48f1417b80 1 -- 172.21.15.96:0/820327345 shutdown_connections 2024-04-25T20:58:09.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 -- 172.21.15.96:0/820327345 wait complete. 2024-04-25T20:58:09.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 -- start start 2024-04-25T20:58:09.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 Processor -- start 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4dee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55563d4e2460 con 0x55563d4d57d0 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55563d3c6b80 con 0x55563d4da580 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55563d4e25e0 con 0x55563d4d60f0 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.221+0000 7f48f1417b80 10 client.0 _ll_register_callbacks cb 0x55563d3c0900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4e1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45724/0 (socket says 172.21.15.96:45724) 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 -- 172.21.15.96:0/671970267 learned_addr learned my addr 172.21.15.96:0/671970267 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48dffff700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4dee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e512b700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 -- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 msgr2=0x55563d4dee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4dee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 -- 172.21.15.96:0/671970267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 msgr2=0x55563d4df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 -- 172.21.15.96:0/671970267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563d3df220 con 0x55563d4d57d0 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e512b700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48e492a700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4e1b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f48d40116c0 tx=0x7f48d40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416c1ff2bd9b9d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48bffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48bffff700 1 -- 172.21.15.96:0/671970267 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4004a30 con 0x55563d4d57d0 2024-04-25T20:58:09.228 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48bffff700 1 -- 172.21.15.96:0/671970267 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f48d4004bd0 con 0x55563d4d57d0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d80a1dc0 con 0x55563d4d57d0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48bffff700 1 -- 172.21.15.96:0/671970267 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f48d4011e40 con 0x55563d4d57d0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48f1417b80 10 client.15363 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.222+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55563d4eccc0 con 0x55563d4d57d0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48bffff700 1 -- 172.21.15.96:0/671970267 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d80a1dc0 con 0x55563d4d57d0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48bffff700 1 -- 172.21.15.96:0/671970267 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f48d402f020 con 0x55563d4d57d0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48f1417b80 10 client.15363 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48f1417b80 10 client.15363 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48f1417b80 20 client.15363 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48f1417b80 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d4ef2e0 0x55563d4f0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48f1417b80 4 client.15363 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55563d4354d0 con 0x55563d4ef2e0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.223+0000 7f48dffff700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d4ef2e0 0x55563d4f0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.224+0000 7f48bffff700 10 client.15363 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.224+0000 7f48dffff700 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d4ef2e0 0x55563d4f0750 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14542dc573c1a218 server_cookie=ff5f67542377f403 in_seq=0 out_seq=0 2024-04-25T20:58:09.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48bffff700 1 -- 172.21.15.96:0/671970267 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55563d4354d0 con 0x55563d4ef2e0 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48bffff700 10 client.15363 handle_command_reply: tid=0 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 client.15363 shutdown 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 msgr2=0x55563d4e1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4e1b00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f48d40116c0 tx=0x7f48d40047c0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 shutdown_connections 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d4ef2e0 0x55563d4f0750 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4e1b00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48bffff700 0 client.15363 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 --2- 172.21.15.96:0/671970267 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4dee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 >> 172.21.15.96:0/671970267 conn(0x55563d402640 msgr2=0x55563d4cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 shutdown_connections 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 1 -- 172.21.15.96:0/671970267 wait complete. 2024-04-25T20:58:09.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.225+0000 7f48f1417b80 20 client.15363 trim_cache size 0 max 16384 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 Processor -- start 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 -- start start 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4da580 0x55563d4f9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55563d4eea90 con 0x55563d4d57d0 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55563d4ef160 con 0x55563d4d60f0 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48f1417b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55563d4ef830 con 0x55563d4da580 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4eb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45728/0 (socket says 172.21.15.96:45728) 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 -- 172.21.15.96:0/3319933025 learned_addr learned my addr 172.21.15.96:0/3319933025 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e512b700 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4da580 0x55563d4f9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 -- 172.21.15.96:0/3319933025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4da580 msgr2=0x55563d4f9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4da580 0x55563d4f9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 -- 172.21.15.96:0/3319933025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 msgr2=0x55563d4e0d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4e0d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e492a700 1 -- 172.21.15.96:0/3319933025 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563d4d5e00 con 0x55563d4d57d0 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.246+0000 7f48e512b700 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4da580 0x55563d4f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.247+0000 7f48e492a700 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4eb4b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f48d400b640 tx=0x7f48d4004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b87831b4d435657 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.247+0000 7f48ca7fc700 1 -- 172.21.15.96:0/3319933025 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4037040 con 0x55563d4d57d0 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.247+0000 7f48ca7fc700 1 -- 172.21.15.96:0/3319933025 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d4023070 con 0x55563d4d57d0 2024-04-25T20:58:09.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.247+0000 7f48ca7fc700 1 -- 172.21.15.96:0/3319933025 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4005260 con 0x55563d4d57d0 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 -- 172.21.15.96:0/3319933025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 msgr2=0x55563d4eb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4eb4b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f48d400b640 tx=0x7f48d4004710 comp rx=0 tx=0).stop 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 -- 172.21.15.96:0/3319933025 shutdown_connections 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4da580 0x55563d4f9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4eb4b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 --2- 172.21.15.96:0/3319933025 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4d60f0 0x55563d4e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.253 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 -- 172.21.15.96:0/3319933025 >> 172.21.15.96:0/3319933025 conn(0x55563d402640 msgr2=0x55563d4e0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 -- 172.21.15.96:0/3319933025 shutdown_connections 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 -- 172.21.15.96:0/3319933025 wait complete. 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 -- start start 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.248+0000 7f48f1417b80 1 Processor -- start 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4f4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55563d4f05d0 con 0x55563d4d57d0 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55563d4eff00 con 0x55563d4da580 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55563d4f0ca0 con 0x55563d4d60f0 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 10 client.0 _ll_register_callbacks cb 0x55563d3c0900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.254 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4f9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45730/0 (socket says 172.21.15.96:45730) 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 -- 172.21.15.96:0/1523420090 learned_addr learned my addr 172.21.15.96:0/1523420090 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48dffff700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e512b700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 -- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 msgr2=0x55563d4f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 -- 172.21.15.96:0/1523420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 msgr2=0x55563d4f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 -- 172.21.15.96:0/1523420090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563d4f5040 con 0x55563d4d57d0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48e492a700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4f9f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f48d40116c0 tx=0x7f48d400b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a21dceca1e267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48bffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48bffff700 1 -- 172.21.15.96:0/1523420090 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48d4037070 con 0x55563d4d57d0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48bffff700 1 -- 172.21.15.96:0/1523420090 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f48d40049c0 con 0x55563d4d57d0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.249+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48d4023070 con 0x55563d4d57d0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48bffff700 1 -- 172.21.15.96:0/1523420090 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f48d404d050 con 0x55563d4d57d0 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 10 client.15369 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:09.255 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55563d5573e0 con 0x55563d4d57d0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48bffff700 1 -- 172.21.15.96:0/1523420090 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48d4023070 con 0x55563d4d57d0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48bffff700 1 -- 172.21.15.96:0/1523420090 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f48d402f040 con 0x55563d4d57d0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 10 client.15369 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 10 client.15369 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 20 client.15369 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d55bcd0 0x55563d55a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 4 client.15369 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "status"}) -- 0x55563d42f570 con 0x55563d55bcd0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.250+0000 7f48dffff700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d55bcd0 0x55563d55a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48bffff700 10 client.15369 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48dffff700 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d55bcd0 0x55563d55a9e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fd5df60052310d5 server_cookie=f39b14064616e965 in_seq=0 out_seq=0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48bffff700 1 -- 172.21.15.96:0/1523420090 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55563d42f570 con 0x55563d55bcd0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48bffff700 10 client.15369 handle_command_reply: tid=0 2024-04-25T20:58:09.256 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 client.15369 shutdown 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 msgr2=0x55563d4f9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4f9f70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f48d40116c0 tx=0x7f48d400b530 comp rx=0 tx=0).stop 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 shutdown_connections 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55563d55bcd0 0x55563d55a9e0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55563d4da580 0x55563d4f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55563d4d57d0 0x55563d4f9f70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48bffff700 0 client.15369 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 --2- 172.21.15.96:0/1523420090 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55563d4d60f0 0x55563d4f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 >> 172.21.15.96:0/1523420090 conn(0x55563d402640 msgr2=0x55563d4e0ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 shutdown_connections 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.251+0000 7f48f1417b80 1 -- 172.21.15.96:0/1523420090 wait complete. 2024-04-25T20:58:09.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48f1417b80 20 client.15369 trim_cache size 0 max 16384 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 1, 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24442, 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 205.935865049, 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 26, 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 94, 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T20:58:09.258 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 245.827723531 2024-04-25T20:58:09.259 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:58:09.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48d00a3fe0 msgr2=0x7f48d00a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48d00a3fe0 0x7f48d00a64a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f48d4005950 tx=0x7f48d400b410 comp rx=0 tx=0).stop 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 msgr2=0x7f48e0102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 0x7f48e0102c90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f48d8009fd0 tx=0x7f48d8004dc0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 shutdown_connections 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48d00a3fe0 0x7f48d00a64a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48e0108ad0 0x7f48e0102c90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48e0101030 0x7f48e0102210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 --2- 172.21.15.96:0/3976893526 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48e01019d0 0x7f48e0102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 >> 172.21.15.96:0/3976893526 conn(0x7f48e0077fb0 msgr2=0x7f48e00ff710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 shutdown_connections 2024-04-25T20:58:09.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:09.252+0000 7f48e6b8e700 1 -- 172.21.15.96:0/3976893526 wait complete. 2024-04-25T20:58:09.984 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:09 smithi096 ceph-mon[29288]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T20:58:10.011 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-25T20:58:10.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:09 smithi097 ceph-mon[32667]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T20:58:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:09 smithi139 ceph-mon[33027]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 21 op/s 2024-04-25T20:58:10.288 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:11.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:11 smithi096 ceph-mon[29288]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T20:58:12.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 -- 172.21.15.96:0/3827438017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4103450 msgr2=0x7f6ca410f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3827438017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4103450 0x7f6ca410f8e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b0a0 tx=0x7f6c9c00b3b0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 -- 172.21.15.96:0/3827438017 shutdown_connections 2024-04-25T20:58:12.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3827438017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4103450 0x7f6ca410f8e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3827438017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4108a80 0x7f6ca4108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3827438017 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca4102a90 0x7f6ca4102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.067+0000 7f6cab69b700 1 -- 172.21.15.96:0/3827438017 >> 172.21.15.96:0/3827438017 conn(0x7f6ca4079bf0 msgr2=0x7f6ca407a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 -- 172.21.15.96:0/3827438017 shutdown_connections 2024-04-25T20:58:12.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 -- 172.21.15.96:0/3827438017 wait complete. 2024-04-25T20:58:12.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 Processor -- start 2024-04-25T20:58:12.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 -- start start 2024-04-25T20:58:12.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 0x7f6ca4074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4103450 0x7f6ca4075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca4108a80 0x7f6ca40757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112c50 con 0x7f6ca4102a90 2024-04-25T20:58:12.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112ad0 con 0x7f6ca4103450 2024-04-25T20:58:12.072 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.068+0000 7f6cab69b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4112dd0 con 0x7f6ca4108a80 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 0x7f6ca4074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 0x7f6ca4074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45746/0 (socket says 172.21.15.96:45746) 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 -- 172.21.15.96:0/3474342289 learned_addr learned my addr 172.21.15.96:0/3474342289 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4103450 0x7f6ca4075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca4108a80 0x7f6ca40757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 -- 172.21.15.96:0/3474342289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca4108a80 msgr2=0x7f6ca40757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca4108a80 0x7f6ca40757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 -- 172.21.15.96:0/3474342289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4103450 msgr2=0x7f6ca4075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4103450 0x7f6ca4075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 -- 172.21.15.96:0/3474342289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca41a7640 con 0x7f6ca4102a90 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4103450 0x7f6ca4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6ca9437700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 0x7f6ca4074d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9a0 tx=0x7f6ca000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6c9a7fc700 1 -- 172.21.15.96:0/3474342289 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca000cbd0 con 0x7f6ca4102a90 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6c9a7fc700 1 -- 172.21.15.96:0/3474342289 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca000cd70 con 0x7f6ca4102a90 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca41a7930 con 0x7f6ca4102a90 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca41a7e70 con 0x7f6ca4102a90 2024-04-25T20:58:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.069+0000 7f6c9a7fc700 1 -- 172.21.15.96:0/3474342289 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca0018680 con 0x7f6ca4102a90 2024-04-25T20:58:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.073+0000 7f6c9a7fc700 1 -- 172.21.15.96:0/3474342289 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f6ca00188e0 con 0x7f6ca4102a90 2024-04-25T20:58:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.073+0000 7f6c9a7fc700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6c900a3f60 0x7f6c900a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.073+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6c900a3f60 0x7f6c900a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.073+0000 7f6c9a7fc700 1 -- 172.21.15.96:0/3474342289 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6ca00d0c20 con 0x7f6ca4102a90 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 Processor -- start 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6c900a3f60 0x7f6c900a6420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ca407a260 tx=0x7f6c9400b2c0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 -- start start 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee93283c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee932c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cee9271ce0 con 0x55cee9328900 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cee9218910 con 0x55cee932cd90 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cee926dca0 con 0x55cee9327fe0 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6ca8c36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee932c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6ca8c36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee932c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45758/0 (socket says 172.21.15.96:45758) 2024-04-25T20:58:12.081 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6ca8c36700 1 -- 172.21.15.96:0/1667692465 learned_addr learned my addr 172.21.15.96:0/1667692465 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.074+0000 7f6ca9437700 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee93283c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6ca8c36700 1 -- 172.21.15.96:0/1667692465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 msgr2=0x55cee93283c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee93283c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6ca8c36700 1 -- 172.21.15.96:0/1667692465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 msgr2=0x55cee932d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6ca8c36700 1 -- 172.21.15.96:0/1667692465 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee91bf880 con 0x55cee9328900 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee932c850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400fed0 tx=0x7f6c94012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13629e23916e364c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6c8f7fe700 1 -- 172.21.15.96:0/1667692465 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94022070 con 0x55cee9328900 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6c8f7fe700 1 -- 172.21.15.96:0/1667692465 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c94011bb0 con 0x55cee9328900 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.075+0000 7f6c8f7fe700 1 -- 172.21.15.96:0/1667692465 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9401d610 con 0x55cee9328900 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/1667692465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 msgr2=0x55cee932c850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.082 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee932c850 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c9400fed0 tx=0x7f6c94012000 comp rx=0 tx=0).stop 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/1667692465 shutdown_connections 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee93283c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/1667692465 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee932c850 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/1667692465 >> 172.21.15.96:0/1667692465 conn(0x55cee924c640 msgr2=0x55cee931a820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/1667692465 shutdown_connections 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/1667692465 wait complete. 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 -- start start 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.076+0000 7f6cb5f24b80 1 Processor -- start 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9327fe0 0x55cee93341f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9328900 0x55cee93314f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 0x55cee9331a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cee9334e20 con 0x55cee932cd90 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cee9210b80 con 0x55cee9327fe0 2024-04-25T20:58:12.083 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cee9334fa0 con 0x55cee9328900 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 0x55cee9331a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6cb5f24b80 10 client.0 _ll_register_callbacks cb 0x55cee920a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 0x55cee9331a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45760/0 (socket says 172.21.15.96:45760) 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 -- 172.21.15.96:0/222196558 learned_addr learned my addr 172.21.15.96:0/222196558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9437700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9327fe0 0x55cee93341f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9328900 0x55cee93314f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 -- 172.21.15.96:0/222196558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9328900 msgr2=0x55cee93314f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9328900 0x55cee93314f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 -- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9327fe0 msgr2=0x55cee93341f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9327fe0 0x55cee93341f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 -- 172.21.15.96:0/222196558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee9229220 con 0x55cee932cd90 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 0x55cee9331a30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b0a0 tx=0x7f6c9c009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f2fbd7fd201dba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.077+0000 7f6c8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6c8cff9700 1 -- 172.21.15.96:0/222196558 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c004c60 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6c8cff9700 1 -- 172.21.15.96:0/222196558 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6c9c004e00 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c94011bb0 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6c8cff9700 1 -- 172.21.15.96:0/222196558 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c9c0074e0 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 10 client.15381 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cee933f6e0 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6c8cff9700 1 -- 172.21.15.96:0/222196558 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c94011bb0 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6c8cff9700 1 -- 172.21.15.96:0/222196558 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6c9c011070 con 0x55cee932cd90 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 10 client.15381 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 10 client.15381 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 20 client.15381 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee9341e00 0x55cee9343270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 4 client.15381 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.078+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cee927f4d0 con 0x55cee9341e00 2024-04-25T20:58:12.085 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.079+0000 7f6ca9437700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee9341e00 0x55cee9343270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.079+0000 7f6ca9437700 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee9341e00 0x55cee9343270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8b3a346b07d001d server_cookie=8e56381543a1529d in_seq=0 out_seq=0 2024-04-25T20:58:12.086 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.079+0000 7f6c8cff9700 10 client.15381 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6c8cff9700 1 -- 172.21.15.96:0/222196558 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cee927f4d0 con 0x55cee9341e00 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6c8cff9700 10 client.15381 handle_command_reply: tid=0 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 client.15381 shutdown 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 msgr2=0x55cee9331a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 0x55cee9331a30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c00b0a0 tx=0x7f6c9c009e20 comp rx=0 tx=0).stop 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 shutdown_connections 2024-04-25T20:58:12.087 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee932cd90 0x55cee9331a30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9327fe0 0x55cee93341f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee9341e00 0x55cee9343270 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/222196558 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9328900 0x55cee93314f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 >> 172.21.15.96:0/222196558 conn(0x55cee924c640 msgr2=0x55cee9319a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6c8cff9700 0 client.15381 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 shutdown_connections 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/222196558 wait complete. 2024-04-25T20:58:12.088 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.080+0000 7f6cb5f24b80 20 client.15381 trim_cache size 0 max 16384 2024-04-25T20:58:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.100+0000 7f6cb5f24b80 1 Processor -- start 2024-04-25T20:58:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.100+0000 7f6cb5f24b80 1 -- start start 2024-04-25T20:58:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.100+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 0x55cee932fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.100+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9328900 0x55cee9330320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.111 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.100+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee932cd90 0x55cee932a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.100+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cee93415b0 con 0x55cee9327fe0 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cee9341c80 con 0x55cee9328900 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cee9342350 con 0x55cee932cd90 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 0x55cee932fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 0x55cee932fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45766/0 (socket says 172.21.15.96:45766) 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 -- 172.21.15.96:0/616342298 learned_addr learned my addr 172.21.15.96:0/616342298 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:12.112 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee932cd90 0x55cee932a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9328900 0x55cee9330320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 -- 172.21.15.96:0/616342298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee932cd90 msgr2=0x55cee932a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee932cd90 0x55cee932a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 -- 172.21.15.96:0/616342298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9328900 msgr2=0x55cee9330320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9328900 0x55cee9330320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 -- 172.21.15.96:0/616342298 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cee932fa60 con 0x55cee9327fe0 2024-04-25T20:58:12.113 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9328900 0x55cee9330320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6ca9437700 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 0x55cee932fde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0010e20 tx=0x7f6ca000e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77e75082073a54f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6c8effd700 1 -- 172.21.15.96:0/616342298 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00b7070 con 0x55cee9327fe0 2024-04-25T20:58:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6c8effd700 1 -- 172.21.15.96:0/616342298 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca009c9f0 con 0x55cee9327fe0 2024-04-25T20:58:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.101+0000 7f6c8effd700 1 -- 172.21.15.96:0/616342298 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00a51d0 con 0x55cee9327fe0 2024-04-25T20:58:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/616342298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 msgr2=0x55cee932fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.114 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 0x55cee932fde0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0010e20 tx=0x7f6ca000e8f0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/616342298 shutdown_connections 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee932cd90 0x55cee932a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9327fe0 0x55cee932fde0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/616342298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee9328900 0x55cee9330320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/616342298 >> 172.21.15.96:0/616342298 conn(0x55cee924c640 msgr2=0x55cee93314b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/616342298 shutdown_connections 2024-04-25T20:58:12.115 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/616342298 wait complete. 2024-04-25T20:58:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.102+0000 7f6cb5f24b80 1 -- start start 2024-04-25T20:58:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 Processor -- start 2024-04-25T20:58:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee9349e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee934a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.116 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cee93430f0 con 0x55cee9328900 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cee9342a20 con 0x55cee932cd90 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cee93437c0 con 0x55cee9327fe0 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6cb5f24b80 10 client.0 _ll_register_callbacks cb 0x55cee920a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6ca8c36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee934a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6ca8c36700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee934a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45778/0 (socket says 172.21.15.96:45778) 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6ca8c36700 1 -- 172.21.15.96:0/229059120 learned_addr learned my addr 172.21.15.96:0/229059120 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:12.117 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.103+0000 7f6ca9437700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee9349e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca8c36700 1 -- 172.21.15.96:0/229059120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 msgr2=0x55cee9349e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee9349e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca8c36700 1 -- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 msgr2=0x55cee932e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca8c36700 1 -- 172.21.15.96:0/229059120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cee932ea90 con 0x55cee9328900 2024-04-25T20:58:12.118 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca9c38700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6ca8c36700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee934a3b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6c94011a00 tx=0x7f6c94011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc50a6d334b24896 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6c8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6c8cff9700 1 -- 172.21.15.96:0/229059120 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c94022040 con 0x55cee9328900 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6c8cff9700 1 -- 172.21.15.96:0/229059120 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6c94005240 con 0x55cee9328900 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ca009c9f0 con 0x55cee9328900 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6c8cff9700 1 -- 172.21.15.96:0/229059120 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6c94005440 con 0x55cee9328900 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6cb5f24b80 10 client.15387 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cee93a95b0 con 0x55cee9328900 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.104+0000 7f6c8cff9700 1 -- 172.21.15.96:0/229059120 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ca009c9f0 con 0x55cee9328900 2024-04-25T20:58:12.119 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6c8cff9700 1 -- 172.21.15.96:0/229059120 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6c94032070 con 0x55cee9328900 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6cb5f24b80 10 client.15387 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6cb5f24b80 10 client.15387 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6cb5f24b80 20 client.15387 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee93adfb0 0x55cee93accc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6cb5f24b80 4 client.15387 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "status"}) -- 0x55cee9279570 con 0x55cee93adfb0 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6ca9437700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee93adfb0 0x55cee93accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6c8cff9700 10 client.15387 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.105+0000 7f6ca9437700 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee93adfb0 0x55cee93accc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd6c4b2babc80e36 server_cookie=9d66c509168cc83 in_seq=0 out_seq=0 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6c8cff9700 1 -- 172.21.15.96:0/229059120 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55cee9279570 con 0x55cee93adfb0 2024-04-25T20:58:12.120 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6c8cff9700 10 client.15387 handle_command_reply: tid=0 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 client.15387 shutdown 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 msgr2=0x55cee934a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee934a3b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6c94011a00 tx=0x7f6c94011bb0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 shutdown_connections 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cee932cd90 0x55cee932e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cee93adfb0 0x55cee93accc0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cee9327fe0 0x55cee9349e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 --2- 172.21.15.96:0/229059120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cee9328900 0x55cee934a3b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 >> 172.21.15.96:0/229059120 conn(0x55cee924c640 msgr2=0x55cee932fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6c8cff9700 0 client.15387 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 shutdown_connections 2024-04-25T20:58:12.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 1 -- 172.21.15.96:0/229059120 wait complete. 2024-04-25T20:58:12.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.106+0000 7f6cb5f24b80 20 client.15387 trim_cache size 0 max 16384 2024-04-25T20:58:12.122 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:58:12.122 INFO:teuthology.orchestra.run.smithi096.stdout: "cluster_fsid": "b713778a-0344-11ef-bc93-c7b262605968", 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "whoami": 0, 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "id": 24443, 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "want_state": "up:active", 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "state": "up:active", 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "fs_name": "cephfs", 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "rank_uptime": 220.712106831, 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "mdsmap_epoch": 26, 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch": 94, 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_epoch_barrier": 93, 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout: "uptime": 238.631709341 2024-04-25T20:58:12.123 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:58:12.123 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:11 smithi097 ceph-mon[32667]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6c900a3f60 msgr2=0x7f6c900a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6c900a3f60 0x7f6c900a6420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ca407a260 tx=0x7f6c9400b2c0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 msgr2=0x7f6ca4074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 0x7f6ca4074d40 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6ca000e9a0 tx=0x7f6ca000ed60 comp rx=0 tx=0).stop 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 shutdown_connections 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6c900a3f60 0x7f6c900a6420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6ca4108a80 0x7f6ca40757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.124 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ca4102a90 0x7f6ca4074d40 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 --2- 172.21.15.96:0/3474342289 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6ca4103450 0x7f6ca4075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:12.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 >> 172.21.15.96:0/3474342289 conn(0x7f6ca4079bf0 msgr2=0x7f6ca41016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:12.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 shutdown_connections 2024-04-25T20:58:12.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:12.107+0000 7f6cab69b700 1 -- 172.21.15.96:0/3474342289 wait complete. 2024-04-25T20:58:12.125 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:11 smithi139 ceph-mon[33027]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-04-25T20:58:12.897 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-25T20:58:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:13 smithi096 ceph-mon[29288]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-25T20:58:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:13 smithi097 ceph-mon[32667]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-25T20:58:14.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:13 smithi139 ceph-mon[33027]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-25T20:58:16.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:16 smithi097 ceph-mon[32667]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T20:58:16.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:16 smithi139 ceph-mon[33027]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T20:58:16.411 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:16 smithi096 ceph-mon[29288]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 25 op/s 2024-04-25T20:58:18.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:18 smithi097 ceph-mon[32667]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T20:58:18.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:18 smithi139 ceph-mon[33027]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T20:58:18.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:18 smithi096 ceph-mon[29288]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 23 op/s 2024-04-25T20:58:20.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:20 smithi097 ceph-mon[32667]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T20:58:20.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:20 smithi139 ceph-mon[33027]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T20:58:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:20 smithi096 ceph-mon[29288]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T20:58:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1231245748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a4106170 msgr2=0x7fe5a410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1231245748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a4106170 0x7fe5a410dd40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ca0 tx=0x7fe59c02a8c0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1231245748 shutdown_connections 2024-04-25T20:58:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1231245748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a4106170 0x7fe5a410dd40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1231245748 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a4101aa0 0x7fe5a4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1231245748 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a41010f0 0x7fe5a41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1231245748 >> 172.21.15.96:0/1231245748 conn(0x7fe5a4077fd0 msgr2=0x7fe5a40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1231245748 shutdown_connections 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.663+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1231245748 wait complete. 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 Processor -- start 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 -- start start 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 0x7fe5a41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a4106170 0x7fe5a4198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a419cf10 0x7fe5a419d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4111130 con 0x7fe5a41010f0 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a4110fb0 con 0x7fe5a4106170 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5aaf76640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41112b0 con 0x7fe5a419cf10 2024-04-25T20:58:20.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 0x7fe5a41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 0x7fe5a41986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45632/0 (socket says 172.21.15.96:45632) 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5a8d12640 1 -- 172.21.15.96:0/1601875307 learned_addr learned my addr 172.21.15.96:0/1601875307 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5a9513640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a419cf10 0x7fe5a419d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.664+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a4106170 0x7fe5a4198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a8d12640 1 -- 172.21.15.96:0/1601875307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a419cf10 msgr2=0x7fe5a419d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a419cf10 0x7fe5a419d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a8d12640 1 -- 172.21.15.96:0/1601875307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a4106170 msgr2=0x7fe5a4198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a4106170 0x7fe5a4198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a8d12640 1 -- 172.21.15.96:0/1601875307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a419da00 con 0x7fe5a41010f0 2024-04-25T20:58:20.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a9513640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a419cf10 0x7fe5a419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 0x7fe5a41986a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a1ffb640 1 -- 172.21.15.96:0/1601875307 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59800cd80 con 0x7fe5a41010f0 2024-04-25T20:58:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.665+0000 7fe5a1ffb640 1 -- 172.21.15.96:0/1601875307 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5980054d0 con 0x7fe5a41010f0 2024-04-25T20:58:20.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.666+0000 7fe5a1ffb640 1 -- 172.21.15.96:0/1601875307 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598010690 con 0x7fe5a41010f0 2024-04-25T20:58:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.666+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a419dcf0 con 0x7fe5a41010f0 2024-04-25T20:58:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.666+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a41a5460 con 0x7fe5a41010f0 2024-04-25T20:58:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.669+0000 7fe5a1ffb640 1 -- 172.21.15.96:0/1601875307 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe598004680 con 0x7fe5a41010f0 2024-04-25T20:58:20.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 Processor -- start 2024-04-25T20:58:20.671 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 -- start start 2024-04-25T20:58:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007b4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.672 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007bf800 0x561a007c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a00719970 con 0x561a007bf800 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a006b64a0 con 0x561a007b4f40 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5add6e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561a00715930 con 0x561a007b4620 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5a1ffb640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe5940a3e50 0x7fe5940a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5a9513640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007b4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5a3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007bf800 0x561a007c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.670+0000 7fe5a9513640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007b4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:36164/0 (socket says 172.21.15.96:36164) 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a9513640 1 -- 172.21.15.96:0/393243023 learned_addr learned my addr 172.21.15.96:0/393243023 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a9513640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe5940a3e50 0x7fe5940a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a3fff640 1 -- 172.21.15.96:0/393243023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 msgr2=0x561a007b4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a1ffb640 1 -- 172.21.15.96:0/1601875307 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe5980d0f50 con 0x7fe5a41010f0 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007b4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.673 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a3fff640 1 -- 172.21.15.96:0/393243023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 msgr2=0x561a007bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a9513640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe5940a3e50 0x7fe5940a6310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009a60 tx=0x7fe59c005f70 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a3fff640 1 -- 172.21.15.96:0/393243023 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a0042c210 con 0x561a007bf800 2024-04-25T20:58:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.671+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:20.674 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.672+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007bf800 0x561a007c1bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe5a406b730 tx=0x7fe590009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e942901280491f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.672+0000 7fe58effd640 1 -- 172.21.15.96:0/393243023 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590011ce0 con 0x561a007bf800 2024-04-25T20:58:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.672+0000 7fe58effd640 1 -- 172.21.15.96:0/393243023 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590016070 con 0x561a007bf800 2024-04-25T20:58:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.672+0000 7fe58effd640 1 -- 172.21.15.96:0/393243023 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe590012450 con 0x561a007bf800 2024-04-25T20:58:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 -- 172.21.15.96:0/393243023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007bf800 msgr2=0x561a007c1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007bf800 0x561a007c1bf0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fe5a406b730 tx=0x7fe590009ed0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.678 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 -- 172.21.15.96:0/393243023 shutdown_connections 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007bf800 0x561a007c1bf0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 --2- 172.21.15.96:0/393243023 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007b4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 -- 172.21.15.96:0/393243023 >> 172.21.15.96:0/393243023 conn(0x561a007b0b30 msgr2=0x561a007b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.673+0000 7fe5add6e740 1 -- 172.21.15.96:0/393243023 shutdown_connections 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.674+0000 7fe5add6e740 1 -- 172.21.15.96:0/393243023 wait complete. 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.674+0000 7fe5add6e740 1 -- start start 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.674+0000 7fe5add6e740 1 Processor -- start 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007bc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4f40 0x561a007bcc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.679 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a007c2430 con 0x561a007b4f40 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561a006b40b0 con 0x561a007bd1c0 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561a007c25b0 con 0x561a007b4620 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007bc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007bc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:36166/0 (socket says 172.21.15.96:36166) 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3256421341 learned_addr learned my addr 172.21.15.96:0/3256421341 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4f40 0x561a007bcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 10 client.0 _ll_register_callbacks cb 0x561a006a1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a9513640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 msgr2=0x561a007cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3256421341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4f40 msgr2=0x561a007bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4f40 0x561a007bcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.680 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3256421341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a004f8620 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4f40 0x561a007bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.675+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe590016070 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007bc740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5980996f0 tx=0x7fe598009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1311b6481365de06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe583fff640 10 client.24922 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe583fff640 1 -- 172.21.15.96:0/3256421341 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980cee60 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe583fff640 1 -- 172.21.15.96:0/3256421341 <== mon.1 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe598098a70 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe583fff640 1 -- 172.21.15.96:0/3256421341 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe59809d420 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe5add6e740 10 client.24922 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.676+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=26}) -- 0x561a007ce990 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe583fff640 1 -- 172.21.15.96:0/3256421341 <== mon.1 v2:172.21.15.139:3300/0 4 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe5980bd0b0 con 0x561a007b4620 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5add6e740 10 client.24922 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5add6e740 10 client.24922 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5add6e740 20 client.24922 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:20.681 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a007d0220 0x561a007d2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5add6e740 4 client.24922 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a00727390 con 0x561a007d0220 2024-04-25T20:58:20.682 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.677+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a007d0220 0x561a007d2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.678+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a007d0220 0x561a007d2670 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa3d035d8e93716 server_cookie=8b0fa3d8b35f8044 in_seq=0 out_seq=0 2024-04-25T20:58:20.683 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.678+0000 7fe583fff640 10 client.24922 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe583fff640 1 -- 172.21.15.96:0/3256421341 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a00727390 con 0x561a007d0220 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe583fff640 10 client.24922 handle_command_reply: tid=0 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 client.24922 shutdown 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 msgr2=0x561a007bc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007bc740 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5980996f0 tx=0x7fe598009310 comp rx=0 tx=0).stop 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 shutdown_connections 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a007d0220 0x561a007d2670 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4f40 0x561a007bcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.679+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3256421341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4620 0x561a007bc740 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.680+0000 7fe583fff640 0 client.24922 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.680+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 >> 172.21.15.96:0/3256421341 conn(0x561a007b0b30 msgr2=0x561a007b6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.681+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 shutdown_connections 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.681+0000 7fe5add6e740 1 -- 172.21.15.96:0/3256421341 wait complete. 2024-04-25T20:58:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.681+0000 7fe5add6e740 20 client.24922 trim_cache size 0 max 16384 2024-04-25T20:58:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 Processor -- start 2024-04-25T20:58:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 -- start start 2024-04-25T20:58:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007bd1c0 0x561a007d7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41affc0 con 0x561a007b4620 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41ad890 con 0x561a007b4f40 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5add6e740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41ab700 con 0x561a007bd1c0 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45672/0 (socket says 172.21.15.96:45672) 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.709+0000 7fe5a8d12640 1 -- 172.21.15.96:0/2748456758 learned_addr learned my addr 172.21.15.96:0/2748456758 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007bd1c0 0x561a007d7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 -- 172.21.15.96:0/2748456758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 msgr2=0x561a007b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 -- 172.21.15.96:0/2748456758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 msgr2=0x561a007d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 -- 172.21.15.96:0/2748456758 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a007c2540 con 0x561a007bd1c0 2024-04-25T20:58:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe5a9513640 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007bd1c0 0x561a007d7f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe59c02aeb0 tx=0x7fe59c041000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56eb23e3f3576ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe58e7fc640 1 -- 172.21.15.96:0/2748456758 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004b40 con 0x561a007bd1c0 2024-04-25T20:58:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe58e7fc640 1 -- 172.21.15.96:0/2748456758 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c036040 con 0x561a007bd1c0 2024-04-25T20:58:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.710+0000 7fe58e7fc640 1 -- 172.21.15.96:0/2748456758 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004040 con 0x561a007bd1c0 2024-04-25T20:58:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.711+0000 7fe5add6e740 1 -- 172.21.15.96:0/2748456758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007bd1c0 msgr2=0x561a007d7f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.711+0000 7fe5add6e740 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007bd1c0 0x561a007d7f80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe59c02aeb0 tx=0x7fe59c041000 comp rx=0 tx=0).stop 2024-04-25T20:58:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.711+0000 7fe5add6e740 1 -- 172.21.15.96:0/2748456758 shutdown_connections 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.711+0000 7fe5add6e740 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007bd1c0 0x561a007d7f80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.711+0000 7fe5add6e740 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007b4f40 0x561a007b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.711+0000 7fe5add6e740 1 --2- 172.21.15.96:0/2748456758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.712+0000 7fe5add6e740 1 -- 172.21.15.96:0/2748456758 >> 172.21.15.96:0/2748456758 conn(0x561a007b0b30 msgr2=0x561a007bdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.712+0000 7fe5add6e740 1 -- 172.21.15.96:0/2748456758 shutdown_connections 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.712+0000 7fe5add6e740 1 -- 172.21.15.96:0/2748456758 wait complete. 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.712+0000 7fe5add6e740 1 -- start start 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.712+0000 7fe5add6e740 1 Processor -- start 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007c7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007c7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007c7c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45684/0 (socket says 172.21.15.96:45684) 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3509148355 learned_addr learned my addr 172.21.15.96:0/3509148355 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:20.717 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4f40 0x561a007c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41abd50 con 0x561a007b4620 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41ad1c0 con 0x561a007bd1c0 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a41ac420 con 0x561a007b4f40 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4f40 0x561a007c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5add6e740 10 client.0 _ll_register_callbacks cb 0x7fe59809cb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3509148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4f40 msgr2=0x561a007c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4f40 0x561a007c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 msgr2=0x561a007c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.713+0000 7fe5a8d12640 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a007c5510 con 0x561a007b4620 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe5a8d12640 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007c7c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe59809ccb0 tx=0x7fe5980d33e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdfa59b6a6522c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe583fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:20.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe583fff640 1 -- 172.21.15.96:0/3509148355 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980a9020 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe583fff640 1 -- 172.21.15.96:0/3509148355 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe5980a5df0 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe59c036040 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe583fff640 1 -- 172.21.15.96:0/3509148355 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe5980cee60 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe5add6e740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561a008357b0 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.714+0000 7fe583fff640 1 -- 172.21.15.96:0/3509148355 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe590016070 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe583fff640 1 -- 172.21.15.96:0/3509148355 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe59809bbe0 con 0x561a007b4620 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5add6e740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5add6e740 10 client.15405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5add6e740 20 client.15405 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a00835ed0 0x561a00838300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5add6e740 4 client.15405 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:58:20.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fe59c036040 con 0x561a00835ed0 2024-04-25T20:58:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a00835ed0 0x561a00838300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe583fff640 10 client.15405 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:20.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.715+0000 7fe5a3fff640 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a00835ed0 0x561a00838300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f227e1fa682e4c70 server_cookie=2db0fc85c6620141 in_seq=0 out_seq=0 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.722+0000 7fe583fff640 1 -- 172.21.15.96:0/3509148355 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7fe59c036040 con 0x561a00835ed0 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.722+0000 7fe583fff640 10 client.15405 handle_command_reply: tid=0 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.722+0000 7fe5add6e740 1 client.15405 shutdown 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.722+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 msgr2=0x561a007c7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.722+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007c7c30 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe59809ccb0 tx=0x7fe5980d33e0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 shutdown_connections 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561a00835ed0 0x561a00838300 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561a007bd1c0 0x561a007c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe583fff640 0 client.15405 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561a007b4f40 0x561a007c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 --2- 172.21.15.96:0/3509148355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a007b4620 0x561a007c7c30 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 >> 172.21.15.96:0/3509148355 conn(0x561a007b0b30 msgr2=0x561a007be800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:20.724 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 shutdown_connections 2024-04-25T20:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 1 -- 172.21.15.96:0/3509148355 wait complete. 2024-04-25T20:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.723+0000 7fe5add6e740 20 client.15405 trim_cache size 0 max 16384 2024-04-25T20:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T20:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T20:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T20:58:20.725 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe5940a3e50 msgr2=0x7fe5940a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe5940a3e50 0x7fe5940a6310 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009a60 tx=0x7fe59c005f70 comp rx=0 tx=0).stop 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 msgr2=0x7fe5a41986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 0x7fe5a41986a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fe59800e9f0 tx=0x7fe59800edf0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 shutdown_connections 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe5940a3e50 0x7fe5940a6310 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe5a419cf10 0x7fe5a419d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5a4106170 0x7fe5a4198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.726 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 --2- 172.21.15.96:0/1601875307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5a41010f0 0x7fe5a41986a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 >> 172.21.15.96:0/1601875307 conn(0x7fe5a4077fd0 msgr2=0x7fe5a4069880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 shutdown_connections 2024-04-25T20:58:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:20.724+0000 7fe5aaf76640 1 -- 172.21.15.96:0/1601875307 wait complete. 2024-04-25T20:58:22.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:22 smithi097 ceph-mon[32667]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T20:58:22.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:22 smithi139 ceph-mon[33027]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T20:58:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:22 smithi096 ceph-mon[29288]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-25T20:58:24.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:24 smithi097 ceph-mon[32667]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:24.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:24 smithi139 ceph-mon[33027]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:24 smithi096 ceph-mon[29288]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:58:26.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:26 smithi097 ceph-mon[32667]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T20:58:26.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:26 smithi139 ceph-mon[33027]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T20:58:26.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:26 smithi096 ceph-mon[29288]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 7 op/s 2024-04-25T20:58:27.898 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-25T20:58:27.899 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.2 2024-04-25T20:58:27.899 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:58:28.173 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:28.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:28 smithi096 ceph-mon[29288]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T20:58:28.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:28 smithi097 ceph-mon[32667]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T20:58:28.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:28 smithi139 ceph-mon[33027]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-04-25T20:58:30.344 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:30 smithi096 ceph-mon[29288]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:30.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:30 smithi097 ceph-mon[32667]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:30.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:30 smithi139 ceph-mon[33027]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.714+0000 7ff23ad5b700 1 -- 172.21.15.96:0/2247872402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2341063f0 msgr2=0x7ff23410dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.714+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/2247872402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2341063f0 0x7ff23410dfc0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b0d0 tx=0x7ff22c00b3e0 comp rx=0 tx=0).stop 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 -- 172.21.15.96:0/2247872402 shutdown_connections 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/2247872402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2341063f0 0x7ff23410dfc0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/2247872402 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff234101360 0x7ff234101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/2247872402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234101d10 0x7ff234105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:30.716 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 -- 172.21.15.96:0/2247872402 >> 172.21.15.96:0/2247872402 conn(0x7ff234077fb0 msgr2=0x7ff2340ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:30.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 -- 172.21.15.96:0/2247872402 shutdown_connections 2024-04-25T20:58:30.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 -- 172.21.15.96:0/2247872402 wait complete. 2024-04-25T20:58:30.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 Processor -- start 2024-04-25T20:58:30.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.715+0000 7ff23ad5b700 1 -- start start 2024-04-25T20:58:30.717 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff23ad5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 0x7ff234108040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff23ad5b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234101d10 0x7ff234108580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff23ad5b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2341063f0 0x7ff234108ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff23ad5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff234110a20 con 0x7ff234101360 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff23ad5b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2341108a0 con 0x7ff234101d10 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff23ad5b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff234110ba0 con 0x7ff2341063f0 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 0x7ff234108040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 0x7ff234108040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36672/0 (socket says 172.21.15.96:36672) 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 -- 172.21.15.96:0/150958180 learned_addr learned my addr 172.21.15.96:0/150958180 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff2392f8700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2341063f0 0x7ff234108ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 -- 172.21.15.96:0/150958180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2341063f0 msgr2=0x7ff234108ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2341063f0 0x7ff234108ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 -- 172.21.15.96:0/150958180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234101d10 msgr2=0x7ff234108580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:30.718 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234101d10 0x7ff234108580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 -- 172.21.15.96:0/150958180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2341ab930 con 0x7ff234101360 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff2392f8700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2341063f0 0x7ff234108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.716+0000 7ff238af7700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 0x7ff234108040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff22400e9c0 tx=0x7ff22400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.717+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff22400cbd0 con 0x7ff234101360 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.717+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2341abc20 con 0x7ff234101360 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.717+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff22400cd70 con 0x7ff234101360 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.717+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff224010430 con 0x7ff234101360 2024-04-25T20:58:30.719 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.717+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2341ac160 con 0x7ff234101360 2024-04-25T20:58:30.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.718+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2341027f0 con 0x7ff234101360 2024-04-25T20:58:30.720 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.719+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff224004750 con 0x7ff234101360 2024-04-25T20:58:30.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.720+0000 7ff231ffb700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff21c0a3fe0 0x7ff21c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:30.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.720+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7ff2240d0db0 con 0x7ff234101360 2024-04-25T20:58:30.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.720+0000 7ff233fff700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff21c0a3fe0 0x7ff21c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:30.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.721+0000 7ff233fff700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff21c0a3fe0 0x7ff21c0a64a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff228005950 tx=0x7ff2280058e0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:30.730 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:30.729+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff22409d280 con 0x7ff234101360 2024-04-25T20:58:31.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.204+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff23402df80 con 0x7ff234101360 2024-04-25T20:58:31.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.206+0000 7ff231ffb700 1 -- 172.21.15.96:0/150958180 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11859 (secure 0 0 0) 0x7ff22409cb80 con 0x7ff234101360 2024-04-25T20:58:31.207 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:58:31.207 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:57:37.187405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T20:58:31.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.208+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff21c0a3fe0 msgr2=0x7ff21c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.209 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.208+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff21c0a3fe0 0x7ff21c0a64a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff228005950 tx=0x7ff2280058e0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.208+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 msgr2=0x7ff234108040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.208+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 0x7ff234108040 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff22400e9c0 tx=0x7ff22400ed80 comp rx=0 tx=0).stop 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 shutdown_connections 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff21c0a3fe0 0x7ff21c0a64a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2341063f0 0x7ff234108ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff234101360 0x7ff234108040 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 --2- 172.21.15.96:0/150958180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff234101d10 0x7ff234108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 >> 172.21.15.96:0/150958180 conn(0x7ff234077fb0 msgr2=0x7ff234105ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 shutdown_connections 2024-04-25T20:58:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:31.209+0000 7ff23ad5b700 1 -- 172.21.15.96:0/150958180 wait complete. 2024-04-25T20:58:31.211 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T20:58:32.057 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24443 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0' 2024-04-25T20:58:32.058 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24443 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0 2024-04-25T20:58:32.327 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:58:32.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:32 smithi097 ceph-mon[32667]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T20:58:32.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:32 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/150958180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:32.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:32 smithi139 ceph-mon[33027]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T20:58:32.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:32 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/150958180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:32 smithi096 ceph-mon[29288]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-04-25T20:58:32.451 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:32 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/150958180' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 -- 172.21.15.96:0/1299719546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00141021c0 msgr2=0x7f0014105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 --2- 172.21.15.96:0/1299719546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00141021c0 0x7f0014105ad0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0004009900 tx=0x7f000402a410 comp rx=0 tx=0).stop 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 -- 172.21.15.96:0/1299719546 shutdown_connections 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 --2- 172.21.15.96:0/1299719546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f001410ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 --2- 172.21.15.96:0/1299719546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f00141021c0 0x7f0014105ad0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 --2- 172.21.15.96:0/1299719546 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f0014101810 0x7f0014101bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.871+0000 7f001a176640 1 -- 172.21.15.96:0/1299719546 >> 172.21.15.96:0/1299719546 conn(0x7f00140773e0 msgr2=0x7f00140fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.874+0000 7f001a176640 1 -- 172.21.15.96:0/1299719546 shutdown_connections 2024-04-25T20:58:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.874+0000 7f001a176640 1 -- 172.21.15.96:0/1299719546 wait complete. 2024-04-25T20:58:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f001a176640 1 Processor -- start 2024-04-25T20:58:33.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f001a176640 1 -- start start 2024-04-25T20:58:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f001a176640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 0x7f0014197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f001a176640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f0014197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f001a176640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f001419bd70 0x7f001419c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f00137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 0x7f0014197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f00137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 0x7f0014197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36694/0 (socket says 172.21.15.96:36694) 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f000bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f0014197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.875+0000 7f000bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f0014197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:46126/0 (socket says 172.21.15.96:46126) 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f00137fe640 1 -- 172.21.15.96:0/2210351349 learned_addr learned my addr 172.21.15.96:0/2210351349 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00141111e0 con 0x7f0014101810 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f0014111060 con 0x7f00141061b0 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f0014111360 con 0x7f001419bd70 2024-04-25T20:58:33.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f0013fff640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f001419bd70 0x7f001419c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f00137fe640 1 -- 172.21.15.96:0/2210351349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f001419bd70 msgr2=0x7f001419c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f00137fe640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f001419bd70 0x7f001419c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f00137fe640 1 -- 172.21.15.96:0/2210351349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 msgr2=0x7f0014197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f00137fe640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f0014197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f00137fe640 1 -- 172.21.15.96:0/2210351349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f001419c860 con 0x7f0014101810 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.876+0000 7f000bfff640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f0014197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.877+0000 7f00137fe640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 0x7f0014197500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b520 tx=0x7efffc00b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.877+0000 7f00117fa640 1 -- 172.21.15.96:0/2210351349 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc01b070 con 0x7f0014101810 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.877+0000 7f00117fa640 1 -- 172.21.15.96:0/2210351349 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efffc00ede0 con 0x7f0014101810 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.877+0000 7f00117fa640 1 -- 172.21.15.96:0/2210351349 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc014340 con 0x7f0014101810 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.877+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f001419cb50 con 0x7f0014101810 2024-04-25T20:58:33.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.878+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0014100760 con 0x7f0014101810 2024-04-25T20:58:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f001cf6b740 1 Processor -- start 2024-04-25T20:58:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f001cf6b740 1 -- start start 2024-04-25T20:58:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63095da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309682c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.883 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63095da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63095da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36700/0 (socket says 172.21.15.96:36700) 2024-04-25T20:58:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f0013fff640 1 -- 172.21.15.96:0/883041430 learned_addr learned my addr 172.21.15.96:0/883041430 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.882+0000 7f00117fa640 1 -- 172.21.15.96:0/2210351349 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7efffc01e070 con 0x7f0014101810 2024-04-25T20:58:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f001cf6b740 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63096abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.884 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6308c2970 con 0x55f63095d620 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f63085f4a0 con 0x55f630968800 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f6308be930 con 0x55f63095df40 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f000bfff640 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63096abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f0013fff640 1 -- 172.21.15.96:0/883041430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 msgr2=0x55f6309682c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f0013fff640 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309682c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f0013fff640 1 -- 172.21.15.96:0/883041430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 msgr2=0x55f63096abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f0013fff640 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63096abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f0013fff640 1 -- 172.21.15.96:0/883041430 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6305d5210 con 0x55f63095d620 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.883+0000 7f00117fa640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f00000a4040 0x7f00000a6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.884+0000 7f0013fff640 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63095da00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f000c009f30 tx=0x7f000c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362f53e9e1a24b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.885 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.884+0000 7f00117fa640 1 -- 172.21.15.96:0/2210351349 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7efffc0d1a50 con 0x7f0014101810 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.884+0000 7f00137fe640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f00000a4040 0x7f00000a6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.884+0000 7f0009ffb640 1 -- 172.21.15.96:0/883041430 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c016070 con 0x55f63095d620 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.884+0000 7f0009ffb640 1 -- 172.21.15.96:0/883041430 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c004d20 con 0x55f63095d620 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.884+0000 7f0009ffb640 1 -- 172.21.15.96:0/883041430 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c0127d0 con 0x55f63095d620 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 msgr2=0x55f63095da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63095da00 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f000c009f30 tx=0x7f000c010800 comp rx=0 tx=0).stop 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f00137fe640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f00000a4040 0x7f00000a6500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efffc0cf140 tx=0x7efffc016a30 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 shutdown_connections 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63096abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 --2- 172.21.15.96:0/883041430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63095da00 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 >> 172.21.15.96:0/883041430 conn(0x55f630959b30 msgr2=0x55f630959f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 shutdown_connections 2024-04-25T20:58:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.885+0000 7f001cf6b740 1 -- 172.21.15.96:0/883041430 wait complete. 2024-04-25T20:58:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 -- start start 2024-04-25T20:58:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 Processor -- start 2024-04-25T20:58:33.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f63095d620 0x55f6309681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309654a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f630968800 0x55f630965a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f63096b430 con 0x55f630968800 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55f63085d0b0 con 0x55f63095d620 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.886+0000 7f001cf6b740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55f63096b5b0 con 0x55f63095df40 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f001cf6b740 10 client.0 _ll_register_callbacks cb 0x55f63084ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f000bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309654a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f000bfff640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309654a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:47292/0 (socket says 172.21.15.96:47292) 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f000bfff640 1 -- 172.21.15.96:0/1536529352 learned_addr learned my addr 172.21.15.96:0/1536529352 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f630968800 0x55f630965a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f63095d620 0x55f6309681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 -- 172.21.15.96:0/1536529352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 msgr2=0x55f6309654a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309654a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.902 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 -- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f63095d620 msgr2=0x55f6309681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f63095d620 0x55f6309681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 -- 172.21.15.96:0/1536529352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6306a1620 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7f0013fff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f630968800 0x55f630965a40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f000c005cc0 tx=0x7f000c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8506ac5cde237c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.887+0000 7efff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7efff2ffd640 1 -- 172.21.15.96:0/1536529352 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c016070 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000c004d20 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7efff2ffd640 1 -- 172.21.15.96:0/1536529352 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f000c0050d0 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7efff2ffd640 1 -- 172.21.15.96:0/1536529352 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f000c0052d0 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7f001cf6b740 10 client.15423 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6309773a0 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.888+0000 7efff2ffd640 1 -- 172.21.15.96:0/1536529352 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c004d20 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7efff2ffd640 1 -- 172.21.15.96:0/1536529352 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f000c032050 con 0x55f630968800 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7f001cf6b740 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:33.903 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7f001cf6b740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7f001cf6b740 20 client.15423 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7f001cf6b740 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f63097aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7f001cf6b740 4 client.15423 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.889+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6308d0390 con 0x55f6309787b0 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.890+0000 7f000bfff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f63097aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.890+0000 7f000bfff640 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f63097aba0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ad39b6dc9e6f8f server_cookie=bb5c3a1e5a6935b1 in_seq=0 out_seq=0 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.890+0000 7efff2ffd640 10 client.15423 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7efff2ffd640 1 -- 172.21.15.96:0/1536529352 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6308d0390 con 0x55f6309787b0 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7efff2ffd640 10 client.15423 handle_command_reply: tid=0 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 client.15423 shutdown 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f630968800 msgr2=0x55f630965a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f630968800 0x55f630965a40 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f000c005cc0 tx=0x7f000c012450 comp rx=0 tx=0).stop 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 shutdown_connections 2024-04-25T20:58:33.904 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f63097aba0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f630968800 0x55f630965a40 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7efff2ffd640 0 client.15423 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f6309654a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 --2- 172.21.15.96:0/1536529352 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f63095d620 0x55f6309681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 >> 172.21.15.96:0/1536529352 conn(0x55f630959b30 msgr2=0x55f63095f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 shutdown_connections 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 1 -- 172.21.15.96:0/1536529352 wait complete. 2024-04-25T20:58:33.905 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.891+0000 7f001cf6b740 20 client.15423 trim_cache size 0 max 16384 2024-04-25T20:58:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 Processor -- start 2024-04-25T20:58:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 -- start start 2024-04-25T20:58:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f630958b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f630967ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.908 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f630958b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f630958b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36710/0 (socket says 172.21.15.96:36710) 2024-04-25T20:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00141addb0 con 0x55f63095d620 2024-04-25T20:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f0013fff640 1 -- 172.21.15.96:0/3369133374 learned_addr learned my addr 172.21.15.96:0/3369133374 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00141ab680 con 0x55f630968800 2024-04-25T20:58:33.909 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00141a9580 con 0x55f63095df40 2024-04-25T20:58:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f000bfff640 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f630967ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.910 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.907+0000 7f0013fff640 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f0013fff640 1 -- 172.21.15.96:0/3369133374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 msgr2=0x55f630967ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f0013fff640 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f630967ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f0013fff640 1 -- 172.21.15.96:0/3369133374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 msgr2=0x55f63097dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f0013fff640 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f0013fff640 1 -- 172.21.15.96:0/3369133374 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f63095ea10 con 0x55f63095d620 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f0013fff640 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f630958b30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f000c012660 tx=0x7f000c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436100621d3d8e0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f00097fa640 1 -- 172.21.15.96:0/3369133374 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c016070 con 0x55f63095d620 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f00097fa640 1 -- 172.21.15.96:0/3369133374 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c024040 con 0x55f63095d620 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.908+0000 7f00097fa640 1 -- 172.21.15.96:0/3369133374 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c005470 con 0x55f63095d620 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 msgr2=0x55f630958b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f630958b30 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f000c012660 tx=0x7f000c010800 comp rx=0 tx=0).stop 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 shutdown_connections 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f630967ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.921 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 --2- 172.21.15.96:0/3369133374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f630958b30 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 >> 172.21.15.96:0/3369133374 conn(0x55f630959b30 msgr2=0x55f630958420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 shutdown_connections 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 -- 172.21.15.96:0/3369133374 wait complete. 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.909+0000 7f001cf6b740 1 -- start start 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 Processor -- start 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63097df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63097df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f63097b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00141a9b40 con 0x55f63095d620 2024-04-25T20:58:33.922 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63097df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36720/0 (socket says 172.21.15.96:36720) 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 -- 172.21.15.96:0/2674110105 learned_addr learned my addr 172.21.15.96:0/2674110105 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f00141aafb0 con 0x55f630968800 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f00141aa210 con 0x55f63095df40 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f001cf6b740 10 client.0 _ll_register_callbacks cb 0x7f000c012550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 -- 172.21.15.96:0/2674110105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 msgr2=0x55f63097b2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f63097b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 -- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 msgr2=0x55f63097b800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097b800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.910+0000 7f0013fff640 1 -- 172.21.15.96:0/2674110105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63097bfd0 con 0x55f63095d620 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7f0013fff640 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63097df30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f000c033770 tx=0x7f000c0337a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f3649e88d856316 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7efff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7efff2ffd640 1 -- 172.21.15.96:0/2674110105 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c016070 con 0x55f63095d620 2024-04-25T20:58:33.923 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7efff2ffd640 1 -- 172.21.15.96:0/2674110105 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f000c005070 con 0x55f63095d620 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000c024040 con 0x55f63095d620 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7efff2ffd640 1 -- 172.21.15.96:0/2674110105 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f000c005910 con 0x55f63095d620 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7f001cf6b740 10 client.15429 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55f6309de940 con 0x55f63095d620 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.911+0000 7efff2ffd640 1 -- 172.21.15.96:0/2674110105 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f000c024040 con 0x55f63095d620 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7efff2ffd640 1 -- 172.21.15.96:0/2674110105 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f000c03c070 con 0x55f63095d620 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7f001cf6b740 10 client.15429 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7f001cf6b740 10 client.15429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7f001cf6b740 20 client.15429 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7f001cf6b740 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f6309e0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7f001cf6b740 4 client.15429 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:58:33.924 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.912+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6308ca200 con 0x55f6309787b0 2024-04-25T20:58:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.918+0000 7f000bfff640 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f6309e0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.918+0000 7efff2ffd640 10 client.15429 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.918+0000 7f000bfff640 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f6309e0f60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5088dc865069eb96 server_cookie=9477fbb1507cf9a5 in_seq=0 out_seq=0 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7efff2ffd640 1 -- 172.21.15.96:0/2674110105 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6308ca200 con 0x55f6309787b0 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7efff2ffd640 10 client.15429 handle_command_reply: tid=0 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 client.15429 shutdown 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 msgr2=0x55f63097df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63097df30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f000c033770 tx=0x7f000c0337a0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 shutdown_connections 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55f6309787b0 0x55f6309e0f60 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7efff2ffd640 0 client.15429 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f630968800 0x55f63097b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55f63095df40 0x55f63097b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.922+0000 7f001cf6b740 1 --2- 172.21.15.96:0/2674110105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f63095d620 0x55f63097df30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 >> 172.21.15.96:0/2674110105 conn(0x55f630959b30 msgr2=0x55f630958cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 shutdown_connections 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001cf6b740 1 -- 172.21.15.96:0/2674110105 wait complete. 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001cf6b740 20 client.15429 trim_cache size 0 max 16384 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T20:58:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T20:58:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f00000a4040 msgr2=0x7f00000a6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f00000a4040 0x7f00000a6500 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efffc0cf140 tx=0x7efffc016a30 comp rx=0 tx=0).stop 2024-04-25T20:58:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 msgr2=0x7f0014197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 0x7f0014197500 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7efffc00b520 tx=0x7efffc00b9e0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 shutdown_connections 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f00000a4040 0x7f00000a6500 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f001419bd70 0x7f001419c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.923+0000 7f001a176640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00141061b0 0x7f0014197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.924+0000 7f001a176640 1 --2- 172.21.15.96:0/2210351349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014101810 0x7f0014197500 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.924+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 >> 172.21.15.96:0/2210351349 conn(0x7f00140773e0 msgr2=0x7f0014104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.924+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 shutdown_connections 2024-04-25T20:58:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:33.924+0000 7f001a176640 1 -- 172.21.15.96:0/2210351349 wait complete. 2024-04-25T20:58:34.105 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:34 smithi096 ceph-mon[29288]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.158+0000 7fbb4f793700 1 -- 172.21.15.96:0/1833891839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 msgr2=0x7fbb4810e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.158+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1833891839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 0x7fbb4810e8d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fbb40009f90 tx=0x7fbb4000b330 comp rx=0 tx=0).stop 2024-04-25T20:58:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 -- 172.21.15.96:0/1833891839 shutdown_connections 2024-04-25T20:58:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1833891839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 0x7fbb4810e8d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1833891839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb48102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1833891839 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb48102d50 0x7fbb481065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 -- 172.21.15.96:0/1833891839 >> 172.21.15.96:0/1833891839 conn(0x7fbb48077fe0 msgr2=0x7fbb480ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 -- 172.21.15.96:0/1833891839 shutdown_connections 2024-04-25T20:58:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.159+0000 7fbb4f793700 1 -- 172.21.15.96:0/1833891839 wait complete. 2024-04-25T20:58:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 Processor -- start 2024-04-25T20:58:34.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 -- start start 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb480ff010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb48102d50 0x7fbb480fd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 0x7fbb480fda80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48111bc0 con 0x7fbb481023a0 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48111a40 con 0x7fbb48106cc0 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4f793700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb48111d40 con 0x7fbb48102d50 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.160+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb480ff010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb480ff010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36738/0 (socket says 172.21.15.96:36738) 2024-04-25T20:58:34.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1313446177 learned_addr learned my addr 172.21.15.96:0/1313446177 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb48102d50 0x7fbb480fd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4dd30700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 0x7fbb480fda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1313446177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb48102d50 msgr2=0x7fbb480fd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb48102d50 0x7fbb480fd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1313446177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 msgr2=0x7fbb480fda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 0x7fbb480fda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1313446177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb481abad0 con 0x7fbb481023a0 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb480ff010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400ec10 tx=0x7fbb4400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb3e7fc700 1 -- 172.21.15.96:0/1313446177 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4400cc90 con 0x7fbb481023a0 2024-04-25T20:58:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb481abd60 con 0x7fbb481023a0 2024-04-25T20:58:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.161+0000 7fbb3e7fc700 1 -- 172.21.15.96:0/1313446177 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4400ce30 con 0x7fbb481023a0 2024-04-25T20:58:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.162+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb481ac240 con 0x7fbb481023a0 2024-04-25T20:58:34.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.162+0000 7fbb3e7fc700 1 -- 172.21.15.96:0/1313446177 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44004020 con 0x7fbb481023a0 2024-04-25T20:58:34.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.166+0000 7fbb3e7fc700 1 -- 172.21.15.96:0/1313446177 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fbb440041c0 con 0x7fbb481023a0 2024-04-25T20:58:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.166+0000 7fbb5a01cb80 1 Processor -- start 2024-04-25T20:58:34.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 -- start start 2024-04-25T20:58:34.168 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb3e7fc700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fbb340a4210 0x7fbb340a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.169 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d691f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d692029c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d692052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d6914ae20 con 0x564d691f8000 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d690f1ac0 con 0x564d691f76e0 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d69146de0 con 0x564d69202f00 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb3e7fc700 1 -- 172.21.15.96:0/1313446177 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fbb44003d50 con 0x7fbb481023a0 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb4dd30700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fbb340a4210 0x7fbb340a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb4cd2e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d692029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb4dd30700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d692052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb4dd30700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d692052f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:47346/0 (socket says 172.21.15.96:47346) 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.167+0000 7fbb4dd30700 1 -- 172.21.15.96:0/1121799516 learned_addr learned my addr 172.21.15.96:0/1121799516 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d691f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1121799516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 msgr2=0x564d692052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d692052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1121799516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 msgr2=0x564d692029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4dd30700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fbb340a4210 0x7fbb340a66d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb40015000 tx=0x7fbb40003ca0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d692029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1121799516 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d690b58d0 con 0x564d691f76e0 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.168+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d692029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.169+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d691f7ac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb440cd910 tx=0x7fbb4409c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a382edcfc38ca164 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.169+0000 7fbb337fe700 1 -- 172.21.15.96:0/1121799516 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440b4020 con 0x564d691f76e0 2024-04-25T20:58:34.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.169+0000 7fbb337fe700 1 -- 172.21.15.96:0/1121799516 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4400ce30 con 0x564d691f76e0 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1121799516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 msgr2=0x564d691f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d691f7ac0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb440cd910 tx=0x7fbb4409c940 comp rx=0 tx=0).stop 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1121799516 shutdown_connections 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d692052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d691f7ac0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1121799516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d692029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1121799516 >> 172.21.15.96:0/1121799516 conn(0x564d69124a20 msgr2=0x564d691f3ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1121799516 shutdown_connections 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.170+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1121799516 wait complete. 2024-04-25T20:58:34.174 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 -- start start 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 Processor -- start 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d69202600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d691ff900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691ffe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d692059f0 con 0x564d691f8000 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564d69104740 con 0x564d691f76e0 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564d69205b70 con 0x564d69202f00 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.171+0000 7fbb5a01cb80 10 client.0 _ll_register_callbacks cb 0x564d690e3ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d69202600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4cd2e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d691ff900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d69202600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:46186/0 (socket says 172.21.15.96:46186) 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 -- 172.21.15.96:0/2987863801 learned_addr learned my addr 172.21.15.96:0/2987863801 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4dd30700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691ffe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.175 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 -- 172.21.15.96:0/2987863801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 msgr2=0x564d691ffe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691ffe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 -- 172.21.15.96:0/2987863801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 msgr2=0x564d691ff900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d691ff900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 -- 172.21.15.96:0/2987863801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d69098a60 con 0x564d691f76e0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d691ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d69202600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb440a48a0 tx=0x7fbb44005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7eec14f4fef93841 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.172+0000 7fbb30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb30ff9700 1 -- 172.21.15.96:0/2987863801 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44062030 con 0x564d691f76e0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4400ce30 con 0x564d691f76e0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb30ff9700 1 -- 172.21.15.96:0/2987863801 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fbb4409c450 con 0x564d691f76e0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb30ff9700 1 -- 172.21.15.96:0/2987863801 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb44084050 con 0x564d691f76e0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb5a01cb80 10 client.24968 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d69210270 con 0x564d691f76e0 2024-04-25T20:58:34.176 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb30ff9700 1 -- 172.21.15.96:0/2987863801 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4400ce30 con 0x564d691f76e0 2024-04-25T20:58:34.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.173+0000 7fbb30ff9700 1 -- 172.21.15.96:0/2987863801 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbb440a4940 con 0x564d691f76e0 2024-04-25T20:58:34.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb5a01cb80 10 client.24968 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:34.178 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb5a01cb80 10 client.24968 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb5a01cb80 20 client.24968 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69212740 0x564d69214b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb5a01cb80 4 client.24968 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d69158610 con 0x564d69212740 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69212740 0x564d69214b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb30ff9700 10 client.24968 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.174+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69212740 0x564d69214b30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=719b90f07224b3cc server_cookie=3564013b1b8a461e in_seq=0 out_seq=0 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb30ff9700 1 -- 172.21.15.96:0/2987863801 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d69158610 con 0x564d69212740 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb30ff9700 10 client.24968 handle_command_reply: tid=0 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 client.24968 shutdown 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 msgr2=0x564d69202600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d69202600 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb440a48a0 tx=0x7fbb44005c70 comp rx=0 tx=0).stop 2024-04-25T20:58:34.179 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 shutdown_connections 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69212740 0x564d69214b30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f76e0 0x564d69202600 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/2987863801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f8000 0x564d691ff900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 >> 172.21.15.96:0/2987863801 conn(0x564d69124a20 msgr2=0x564d691f2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.175+0000 7fbb30ff9700 0 client.24968 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.176+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 shutdown_connections 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.176+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/2987863801 wait complete. 2024-04-25T20:58:34.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.176+0000 7fbb5a01cb80 20 client.24968 trim_cache size 0 max 16384 2024-04-25T20:58:34.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.195+0000 7fbb5a01cb80 1 Processor -- start 2024-04-25T20:58:34.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.195+0000 7fbb5a01cb80 1 -- start start 2024-04-25T20:58:34.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d692042f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69204830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691fcbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b6040 con 0x564d691f76e0 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b38d0 con 0x564d691f8000 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b1760 con 0x564d69202f00 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d692042f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d692042f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36764/0 (socket says 172.21.15.96:36764) 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1351597185 learned_addr learned my addr 172.21.15.96:0/1351597185 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4dd30700 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691fcbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1351597185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 msgr2=0x564d691fcbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691fcbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1351597185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 msgr2=0x564d69204830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69204830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 -- 172.21.15.96:0/1351597185 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d69200150 con 0x564d691f76e0 2024-04-25T20:58:34.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.196+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d692042f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb4409c180 tx=0x7fbb44005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3788bb6c856f6f20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.197+0000 7fbb32ffd700 1 -- 172.21.15.96:0/1351597185 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44062070 con 0x564d691f76e0 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.197+0000 7fbb32ffd700 1 -- 172.21.15.96:0/1351597185 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4400ce30 con 0x564d691f76e0 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.197+0000 7fbb32ffd700 1 -- 172.21.15.96:0/1351597185 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb440bb050 con 0x564d691f76e0 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1351597185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 msgr2=0x564d692042f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d692042f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fbb4409c180 tx=0x7fbb44005c00 comp rx=0 tx=0).stop 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1351597185 shutdown_connections 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d691fcbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d692042f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/1351597185 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69204830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1351597185 >> 172.21.15.96:0/1351597185 conn(0x564d69124a20 msgr2=0x564d691ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1351597185 shutdown_connections 2024-04-25T20:58:34.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/1351597185 wait complete. 2024-04-25T20:58:34.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 -- start start 2024-04-25T20:58:34.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.198+0000 7fbb5a01cb80 1 Processor -- start 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d69217830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69217d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d69215d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b1db0 con 0x564d691f76e0 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b3200 con 0x564d691f8000 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fbb481b2460 con 0x564d69202f00 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d69217830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb5a01cb80 10 client.0 _ll_register_callbacks cb 0x564d690e3ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb4d52f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d69217830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36768/0 (socket says 172.21.15.96:36768) 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb4d52f700 1 -- 172.21.15.96:0/344859414 learned_addr learned my addr 172.21.15.96:0/344859414 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.199+0000 7fbb4dd30700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d69215d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4d52f700 1 -- 172.21.15.96:0/344859414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 msgr2=0x564d69215d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69217d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d69215d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4d52f700 1 -- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 msgr2=0x564d69217d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69217d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4d52f700 1 -- 172.21.15.96:0/344859414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d69216440 con 0x564d691f76e0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4d52f700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d69217830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb440bb270 tx=0x7fbb440a05a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d278edeeb8c2b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb30ff9700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.200+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69217d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.201+0000 7fbb30ff9700 1 -- 172.21.15.96:0/344859414 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb44062040 con 0x564d691f76e0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.201+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb4400ce30 con 0x564d691f76e0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.201+0000 7fbb30ff9700 1 -- 172.21.15.96:0/344859414 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fbb4409b640 con 0x564d691f76e0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.201+0000 7fbb30ff9700 1 -- 172.21.15.96:0/344859414 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fbb440b4070 con 0x564d691f76e0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.201+0000 7fbb5a01cb80 10 client.15447 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.201+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564d69216440 con 0x564d691f76e0 2024-04-25T20:58:34.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb30ff9700 1 -- 172.21.15.96:0/344859414 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4400ce30 con 0x564d691f76e0 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb30ff9700 1 -- 172.21.15.96:0/344859414 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fbb440cdc20 con 0x564d691f76e0 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb5a01cb80 10 client.15447 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb5a01cb80 10 client.15447 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb5a01cb80 20 client.15447 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69279660 0x564d6927ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb5a01cb80 4 client.15447 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}] 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}) -- 0x564d691526b0 con 0x564d69279660 2024-04-25T20:58:34.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69279660 0x564d6927ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:34.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb30ff9700 10 client.15447 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:34.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.202+0000 7fbb4cd2e700 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69279660 0x564d6927ba80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5cdd47fc13f629e server_cookie=229861f27a2c0e62 in_seq=0 out_seq=0 2024-04-25T20:58:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.214+0000 7fbb30ff9700 1 -- 172.21.15.96:0/344859414 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x564d691526b0 con 0x564d69279660 2024-04-25T20:58:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.214+0000 7fbb30ff9700 10 client.15447 handle_command_reply: tid=0 2024-04-25T20:58:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.214+0000 7fbb5a01cb80 1 client.15447 shutdown 2024-04-25T20:58:34.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.214+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 msgr2=0x564d69217830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.214+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d69217830 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fbb440bb270 tx=0x7fbb440a05a0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 shutdown_connections 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564d69279660 0x564d6927ba80 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564d69202f00 0x564d69215d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb30ff9700 0 client.15447 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d691f76e0 0x564d69217830 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 --2- 172.21.15.96:0/344859414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564d691f8000 0x564d69217d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 >> 172.21.15.96:0/344859414 conn(0x564d69124a20 msgr2=0x564d6920de40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 shutdown_connections 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 1 -- 172.21.15.96:0/344859414 wait complete. 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.215+0000 7fbb5a01cb80 20 client.15447 trim_cache size 0 max 16384 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 26, 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "leader": 24443, 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "set_version": 2, 2024-04-25T20:58:34.217 INFO:teuthology.orchestra.run.smithi096.stdout: "sets": { 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "fd1653f2": { 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "version": 2, 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age_ref": 0.0, 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "QUIESCED", 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "timeout": 60.0, 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "expiration": 86.2, 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "members": { 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "file:/": { 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "excluded": false, 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T20:58:34.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "QUIESCED", 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:58:34.219 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fbb340a4210 msgr2=0x7fbb340a66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fbb340a4210 0x7fbb340a66d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbb40015000 tx=0x7fbb40003ca0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 msgr2=0x7fbb480ff010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb480ff010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fbb4400ec10 tx=0x7fbb4400ef20 comp rx=0 tx=0).stop 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 shutdown_connections 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fbb340a4210 0x7fbb340a66d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb48106cc0 0x7fbb480fda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb481023a0 0x7fbb480ff010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 --2- 172.21.15.96:0/1313446177 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fbb48102d50 0x7fbb480fd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 >> 172.21.15.96:0/1313446177 conn(0x7fbb48077fe0 msgr2=0x7fbb48101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 shutdown_connections 2024-04-25T20:58:34.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:58:34.216+0000 7fbb4f793700 1 -- 172.21.15.96:0/1313446177 wait complete. 2024-04-25T20:58:34.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:34 smithi097 ceph-mon[32667]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:34.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:34 smithi139 ceph-mon[33027]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-04-25T20:58:34.904 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fd1653f2', quiesce took 7.0 seconds. Will release after: 26.2 2024-04-25T20:58:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:36 smithi097 ceph-mon[32667]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-25T20:58:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:36 smithi139 ceph-mon[33027]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-25T20:58:36.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:36 smithi096 ceph-mon[29288]: pgmap v31: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 8 op/s 2024-04-25T20:58:38.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:38 smithi097 ceph-mon[32667]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T20:58:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:38 smithi139 ceph-mon[33027]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T20:58:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:38 smithi096 ceph-mon[29288]: pgmap v32: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 938 B/s wr, 6 op/s 2024-04-25T20:58:40.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:40 smithi097 ceph-mon[32667]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T20:58:40.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:40 smithi139 ceph-mon[33027]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T20:58:40.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:40 smithi096 ceph-mon[29288]: pgmap v33: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T20:58:42.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:42 smithi139 ceph-mon[33027]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T20:58:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:42 smithi096 ceph-mon[29288]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T20:58:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:42 smithi097 ceph-mon[32667]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 938 B/s wr, 7 op/s 2024-04-25T20:58:44.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:44 smithi139 ceph-mon[33027]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T20:58:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:44 smithi096 ceph-mon[29288]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T20:58:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:44 smithi097 ceph-mon[32667]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 938 B/s wr, 5 op/s 2024-04-25T20:58:45.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:45 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:45.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:45 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:45.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:45 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:58:46.032 INFO:tasks.workunit.client.0.smithi096.stderr:Note: switching to 'e221b808d285b9d02e54cc10c53c84188e8e41e3'. 2024-04-25T20:58:46.032 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.032 INFO:tasks.workunit.client.0.smithi096.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-25T20:58:46.032 INFO:tasks.workunit.client.0.smithi096.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr:state without impacting any branches by switching back to a branch. 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: git switch -c 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr:Or undo this operation with: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: git switch - 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr: 2024-04-25T20:58:46.033 INFO:tasks.workunit.client.0.smithi096.stderr:HEAD is now at e221b808d28 qa/suites/fs: add quiescer to the fs suite 2024-04-25T20:58:46.035 DEBUG:teuthology.orchestra.run.smithi096:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-25T20:58:46.069 INFO:tasks.workunit.client.0.smithi096.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-25T20:58:46.071 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T20:58:46.071 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-25T20:58:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:46 smithi096 ceph-mon[29288]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T20:58:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:58:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:46 smithi097 ceph-mon[32667]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T20:58:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:46.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:58:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:46 smithi139 ceph-mon[33027]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1023 B/s wr, 7 op/s 2024-04-25T20:58:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:58:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:58:46.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:58:46.779 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-25T20:58:46.835 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-25T20:58:46.875 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-25T20:58:46.877 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T20:58:46.877 INFO:tasks.workunit.client.0.smithi096.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-25T20:58:46.915 INFO:tasks.workunit.client.0.smithi096.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-25T20:58:46.918 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T20:58:46.919 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-25T20:58:46.974 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-04-25T20:58:46.975 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-04-25T20:58:46.975 DEBUG:teuthology.orchestra.run.smithi096:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=e221b808d285b9d02e54cc10c53c84188e8e41e3 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 -- 172.21.15.96:0/117288286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c90102b80 msgr2=0x7f9c90106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 --2- 172.21.15.96:0/117288286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c90102b80 0x7f9c90106bc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c009900 tx=0x7f9c7c02a410 comp rx=0 tx=0).stop 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 -- 172.21.15.96:0/117288286 shutdown_connections 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 --2- 172.21.15.96:0/117288286 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 0x7f9c9010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 --2- 172.21.15.96:0/117288286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c90102b80 0x7f9c90106bc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 --2- 172.21.15.96:0/117288286 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 0x7f9c901025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 -- 172.21.15.96:0/117288286 >> 172.21.15.96:0/117288286 conn(0x7f9c900793e0 msgr2=0x7f9c900797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 -- 172.21.15.96:0/117288286 shutdown_connections 2024-04-25T20:58:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 -- 172.21.15.96:0/117288286 wait complete. 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 Processor -- start 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 -- start start 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.013+0000 7f9c96a72640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 0x7f9c90198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c96a72640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 0x7f9c90198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c96a72640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 0x7f9c9019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c96a72640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c90112240 con 0x7f9c9019cfc0 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c96a72640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901120c0 con 0x7f9c901021d0 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c96a72640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901123c0 con 0x7f9c901072a0 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 0x7f9c9019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 0x7f9c9019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36766/0 (socket says 172.21.15.96:36766) 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 -- 172.21.15.96:0/2957359619 learned_addr learned my addr 172.21.15.96:0/2957359619 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 0x7f9c90198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 -- 172.21.15.96:0/2957359619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 msgr2=0x7f9c90198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 0x7f9c90198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 -- 172.21.15.96:0/2957359619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 msgr2=0x7f9c90198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9480e640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 0x7f9c90198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 0x7f9c90198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 -- 172.21.15.96:0/2957359619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c9019dab0 con 0x7f9c9019cfc0 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 0x7f9c90198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9480e640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 0x7f9c90198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c9500f640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 0x7f9c9019d470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800b9e0 tx=0x7f9c8800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c8dffb640 1 -- 172.21.15.96:0/2957359619 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88019070 con 0x7f9c9019cfc0 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c8dffb640 1 -- 172.21.15.96:0/2957359619 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c88014440 con 0x7f9c9019cfc0 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c8dffb640 1 -- 172.21.15.96:0/2957359619 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880075d0 con 0x7f9c9019cfc0 2024-04-25T20:58:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.014+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c9019dda0 con 0x7f9c9019cfc0 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.015+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c901a5560 con 0x7f9c9019cfc0 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.016+0000 7f9c8dffb640 1 -- 172.21.15.96:0/2957359619 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f9c8801c070 con 0x7f9c9019cfc0 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 Processor -- start 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 -- start start 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c8dffb640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9c800a3f70 0x7f9c800a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6620 0x555c1cfb6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1cfc12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfc3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c1cf1b970 con 0x555c1cfc1800 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555c1ceb84a0 con 0x555c1cfb6620 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c99783740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555c1cf17930 con 0x555c1cfb6f40 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c8dffb640 1 -- 172.21.15.96:0/2957359619 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f9c880cfeb0 con 0x7f9c9019cfc0 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9c800a3f70 0x7f9c800a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9500f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfc3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9500f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfc3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36770/0 (socket says 172.21.15.96:36770) 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9500f640 1 -- 172.21.15.96:0/438640256 learned_addr learned my addr 172.21.15.96:0/438640256 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9480e640 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6620 0x555c1cfb6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9500f640 1 -- 172.21.15.96:0/438640256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 msgr2=0x555c1cfc12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9500f640 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1cfc12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.017+0000 7f9c9500f640 1 -- 172.21.15.96:0/438640256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6620 msgr2=0x555c1cfb6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c9500f640 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6620 0x555c1cfb6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c9500f640 1 -- 172.21.15.96:0/438640256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1cc2e210 con 0x555c1cfc1800 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c9500f640 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfc3bf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c880d0190 tx=0x7f9c88004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b859a6bf2f938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9c800a3f70 0x7f9c800a6430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000c00 tx=0x7f9c7c0095f0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c7affd640 1 -- 172.21.15.96:0/438640256 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880a0140 con 0x555c1cfc1800 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c7affd640 1 -- 172.21.15.96:0/438640256 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c880a4a00 con 0x555c1cfc1800 2024-04-25T20:58:47.025 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.018+0000 7f9c7affd640 1 -- 172.21.15.96:0/438640256 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c88060110 con 0x555c1cfc1800 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 -- 172.21.15.96:0/438640256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 msgr2=0x555c1cfc3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfc3bf0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9c880d0190 tx=0x7f9c88004b50 comp rx=0 tx=0).stop 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 -- 172.21.15.96:0/438640256 shutdown_connections 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfc3bf0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1cfc12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 --2- 172.21.15.96:0/438640256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6620 0x555c1cfb6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 -- 172.21.15.96:0/438640256 >> 172.21.15.96:0/438640256 conn(0x555c1cfb2b30 msgr2=0x555c1cfb2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 -- 172.21.15.96:0/438640256 shutdown_connections 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 -- 172.21.15.96:0/438640256 wait complete. 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 -- start start 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.019+0000 7f9c99783740 1 Processor -- start 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfc11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6f40 0x555c1cfbe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfbea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c1cfc4430 con 0x555c1cfc1800 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555c1ceb60b0 con 0x555c1cfb6f40 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555c1cfc45b0 con 0x555c1cfb6620 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 10 client.0 _ll_register_callbacks cb 0x555c1cea3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfbea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfbea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36776/0 (socket says 172.21.15.96:36776) 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 -- 172.21.15.96:0/873035073 learned_addr learned my addr 172.21.15.96:0/873035073 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9480e640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfc11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6f40 0x555c1cfbe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 -- 172.21.15.96:0/873035073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 msgr2=0x555c1cfc11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfc11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 -- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6f40 msgr2=0x555c1cfbe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6f40 0x555c1cfbe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 -- 172.21.15.96:0/873035073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1ccfa620 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6f40 0x555c1cfbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9480e640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfc11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c9500f640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfbea40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9c88098760 tx=0x7f9c880603a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116a6207f8eac1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c6ffff640 1 -- 172.21.15.96:0/873035073 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c880145f0 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c6ffff640 1 -- 172.21.15.96:0/873035073 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f9c880a4780 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c880a4a00 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c6ffff640 1 -- 172.21.15.96:0/873035073 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c880a0140 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.020+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c1cfd03a0 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c6ffff640 1 -- 172.21.15.96:0/873035073 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c880a4a00 con 0x555c1cfc1800 2024-04-25T20:58:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c6ffff640 1 -- 172.21.15.96:0/873035073 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9c88097ae0 con 0x555c1cfc1800 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c99783740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c99783740 10 client.15459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c99783740 20 client.15459 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c99783740 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1cfd16f0 0x555c1cfd3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c99783740 4 client.15459 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c1cf29390 con 0x555c1cfd16f0 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.021+0000 7f9c9480e640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1cfd16f0 0x555c1cfd3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c9480e640 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1cfd16f0 0x555c1cfd3ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e0e92bde54f6e78 server_cookie=7f70b21b81b9596a in_seq=0 out_seq=0 2024-04-25T20:58:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c6ffff640 10 client.15459 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c6ffff640 1 -- 172.21.15.96:0/873035073 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555c1cf29390 con 0x555c1cfd16f0 2024-04-25T20:58:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c6ffff640 10 client.15459 handle_command_reply: tid=0 2024-04-25T20:58:47.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c99783740 1 client.15459 shutdown 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 msgr2=0x555c1cfbea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c99783740 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfbea40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f9c88098760 tx=0x7f9c880603a0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 shutdown_connections 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.022+0000 7f9c99783740 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1cfd16f0 0x555c1cfd3ae0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfc1800 0x555c1cfbea40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c6ffff640 0 client.15459 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfb6f40 0x555c1cfbe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 1 --2- 172.21.15.96:0/873035073 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfc11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 >> 172.21.15.96:0/873035073 conn(0x555c1cfb2b30 msgr2=0x555c1cfb88b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 shutdown_connections 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 1 -- 172.21.15.96:0/873035073 wait complete. 2024-04-25T20:58:47.031 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.023+0000 7f9c99783740 20 client.15459 trim_cache size 0 max 16384 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.037+0000 7f9c99783740 1 Processor -- start 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.037+0000 7f9c99783740 1 -- start start 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.037+0000 7f9c99783740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfd6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.037+0000 7f9c99783740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 0x555c1cfd7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.037+0000 7f9c99783740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1cfbb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c99783740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901b0280 con 0x555c1cfb6f40 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c99783740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901adb50 con 0x555c1cfc1800 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c99783740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901ab9c0 con 0x555c1cfb6620 2024-04-25T20:58:47.043 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 0x555c1cfd7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 0x555c1cfd7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36778/0 (socket says 172.21.15.96:36778) 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 -- 172.21.15.96:0/2909654350 learned_addr learned my addr 172.21.15.96:0/2909654350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c9480e640 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfd6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 -- 172.21.15.96:0/2909654350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 msgr2=0x555c1cfd6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfd6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c9500f640 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1cfbb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 -- 172.21.15.96:0/2909654350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 msgr2=0x555c1cfbb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1cfbb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 -- 172.21.15.96:0/2909654350 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c1cfc3f20 con 0x555c1cfb6f40 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 0x555c1cfd7050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9c9006b630 tx=0x7f9c7c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbeee82794086fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c7a7fc640 1 -- 172.21.15.96:0/2909654350 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c035050 con 0x555c1cfb6f40 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c7a7fc640 1 -- 172.21.15.96:0/2909654350 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7c04f070 con 0x555c1cfb6f40 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.038+0000 7f9c7a7fc640 1 -- 172.21.15.96:0/2909654350 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c7c04a6b0 con 0x555c1cfb6f40 2024-04-25T20:58:47.044 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 -- 172.21.15.96:0/2909654350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 msgr2=0x555c1cfd7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 0x555c1cfd7050 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9c9006b630 tx=0x7f9c7c03e000 comp rx=0 tx=0).stop 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 -- 172.21.15.96:0/2909654350 shutdown_connections 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1cfbb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6f40 0x555c1cfd7050 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 --2- 172.21.15.96:0/2909654350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6620 0x555c1cfd6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 -- 172.21.15.96:0/2909654350 >> 172.21.15.96:0/2909654350 conn(0x555c1cfb2b30 msgr2=0x555c1cfcd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 -- 172.21.15.96:0/2909654350 shutdown_connections 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 -- 172.21.15.96:0/2909654350 wait complete. 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 -- start start 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.039+0000 7f9c99783740 1 Processor -- start 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 0x555c1d02ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1d02bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1d030920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.045 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901ac010 con 0x555c1cfb6620 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901ad480 con 0x555c1cfc1800 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9c901ac6e0 con 0x555c1cfb6f40 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 10 client.0 _ll_register_callbacks cb 0x7f9c8809be50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 0x555c1d02ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 0x555c1d02ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36784/0 (socket says 172.21.15.96:36784) 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 -- 172.21.15.96:0/1209220986 learned_addr learned my addr 172.21.15.96:0/1209220986 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1d02bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9500f640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1d030920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 -- 172.21.15.96:0/1209220986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 msgr2=0x555c1d02bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1d02bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 -- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 msgr2=0x555c1d030920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1d030920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.046 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 -- 172.21.15.96:0/1209220986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c1d030f60 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9500f640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1d030920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c9480e640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 0x555c1d02ba50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x555c1cfc45f0 tx=0x7f9c8400fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff20c36603798551 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c6ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c6ffff640 1 -- 172.21.15.96:0/1209220986 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c8401c910 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c6ffff640 1 -- 172.21.15.96:0/1209220986 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f9c84025470 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.040+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c7c04f070 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c6ffff640 1 -- 172.21.15.96:0/1209220986 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f9c84025670 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x555c1d038380 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c6ffff640 1 -- 172.21.15.96:0/1209220986 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c7c04f070 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c6ffff640 1 -- 172.21.15.96:0/1209220986 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f9c8402d020 con 0x555c1cfb6620 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:58:47.047 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 10 client.15465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 20 client.15465 populate_metadata read hostname 'smithi096' 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1d038ab0 0x555c1d03aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 4 client.15465 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c1cf23200 con 0x555c1d038ab0 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.041+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1d038ab0 0x555c1d03aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.042+0000 7f9c8ffff640 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1d038ab0 0x555c1d03aed0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad1ca96334f95e60 server_cookie=71f08838053dc54e in_seq=0 out_seq=0 2024-04-25T20:58:47.048 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.042+0000 7f9c6ffff640 10 client.15465 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c6ffff640 1 -- 172.21.15.96:0/1209220986 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555c1cf23200 con 0x555c1d038ab0 2024-04-25T20:58:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c6ffff640 10 client.15465 handle_command_reply: tid=0 2024-04-25T20:58:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 client.15465 shutdown 2024-04-25T20:58:47.050 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 msgr2=0x555c1d02ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 0x555c1d02ba50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x555c1cfc45f0 tx=0x7f9c8400fae0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 shutdown_connections 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555c1d038ab0 0x555c1d03aed0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555c1cfc1800 0x555c1d030920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555c1cfb6f40 0x555c1d02bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 --2- 172.21.15.96:0/1209220986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c1cfb6620 0x555c1d02ba50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c6ffff640 0 client.15465 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 >> 172.21.15.96:0/1209220986 conn(0x555c1cfb2b30 msgr2=0x555c1cfb1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 shutdown_connections 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 1 -- 172.21.15.96:0/1209220986 wait complete. 2024-04-25T20:58:47.051 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.049+0000 7f9c99783740 20 client.15465 trim_cache size 0 max 16384 2024-04-25T20:58:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T20:58:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T20:58:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T20:58:47.052 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9c800a3f70 msgr2=0x7f9c800a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9c800a3f70 0x7f9c800a6430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c7c000c00 tx=0x7f9c7c0095f0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 msgr2=0x7f9c9019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 0x7f9c9019d470 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9c8800b9e0 tx=0x7f9c8800bea0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 shutdown_connections 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9c800a3f70 0x7f9c800a6430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c9019cfc0 0x7f9c9019d470 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9c901072a0 0x7f9c90198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 --2- 172.21.15.96:0/2957359619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c901021d0 0x7f9c90198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 >> 172.21.15.96:0/2957359619 conn(0x7f9c900793e0 msgr2=0x7f9c900ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 shutdown_connections 2024-04-25T20:58:47.053 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:58:47.050+0000 7f9c96a72640 1 -- 172.21.15.96:0/2957359619 wait complete. 2024-04-25T20:58:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:48 smithi096 ceph-mon[29288]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:48 smithi097 ceph-mon[32667]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:48 smithi139 ceph-mon[33027]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:50.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:50 smithi096 ceph-mon[29288]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:50.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:50 smithi097 ceph-mon[32667]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:50 smithi139 ceph-mon[33027]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T20:58:52.416 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:52 smithi097 ceph-mon[32667]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:58:52.424 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:52 smithi139 ceph-mon[33027]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:58:52.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:52 smithi096 ceph-mon[29288]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:58:54.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:54 smithi096 ceph-mon[29288]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T20:58:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:54 smithi097 ceph-mon[32667]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T20:58:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:54 smithi139 ceph-mon[33027]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T20:58:56.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:56 smithi096 ceph-mon[29288]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:58:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:56 smithi097 ceph-mon[32667]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:58:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:56 smithi139 ceph-mon[33027]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:58:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:58:58 smithi096 ceph-mon[29288]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T20:58:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:58:58 smithi097 ceph-mon[32667]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T20:58:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:58:58 smithi139 ceph-mon[33027]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-04-25T20:59:00.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 -- 172.21.15.96:0/1970915347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 msgr2=0x7fe7601014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 --2- 172.21.15.96:0/1970915347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe7601014d0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009900 tx=0x7fe75c02a410 comp rx=0 tx=0).stop 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 -- 172.21.15.96:0/1970915347 shutdown_connections 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 --2- 172.21.15.96:0/1970915347 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe760106170 0x7fe76010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 --2- 172.21.15.96:0/1970915347 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760101aa0 0x7fe760105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 --2- 172.21.15.96:0/1970915347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe7601014d0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 -- 172.21.15.96:0/1970915347 >> 172.21.15.96:0/1970915347 conn(0x7fe760077fd0 msgr2=0x7fe7600783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 -- 172.21.15.96:0/1970915347 shutdown_connections 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.149+0000 7fe768136640 1 -- 172.21.15.96:0/1970915347 wait complete. 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 Processor -- start 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 -- start start 2024-04-25T20:59:00.153 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe760102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760106170 0x7fe760103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7601035b0 0x7fe7601a6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe760111130 con 0x7fe7601010f0 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe760110fb0 con 0x7fe760106170 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe768136640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601112b0 con 0x7fe7601035b0 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe760102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe760102b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42764/0 (socket says 172.21.15.96:42764) 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe765ed2640 1 -- 172.21.15.96:0/3124569141 learned_addr learned my addr 172.21.15.96:0/3124569141 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.150+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760106170 0x7fe760103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe765ed2640 1 -- 172.21.15.96:0/3124569141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7601035b0 msgr2=0x7fe7601a6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7601035b0 0x7fe7601a6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7601035b0 0x7fe7601a6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.154 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe765ed2640 1 -- 172.21.15.96:0/3124569141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760106170 msgr2=0x7fe760103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760106170 0x7fe760103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe765ed2640 1 -- 172.21.15.96:0/3124569141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7601a6f90 con 0x7fe7601010f0 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760106170 0x7fe760103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7601035b0 0x7fe7601a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe760102b30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005ab0 tx=0x7fe75c003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe756ffd640 1 -- 172.21.15.96:0/3124569141 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c03f070 con 0x7fe7601010f0 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe756ffd640 1 -- 172.21.15.96:0/3124569141 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c031d60 con 0x7fe7601010f0 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7601a7220 con 0x7fe7601010f0 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe756ffd640 1 -- 172.21.15.96:0/3124569141 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c035030 con 0x7fe7601010f0 2024-04-25T20:59:00.155 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.151+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7601a7710 con 0x7fe7601010f0 2024-04-25T20:59:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.153+0000 7fe756ffd640 1 -- 172.21.15.96:0/3124569141 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe75c042070 con 0x7fe7601010f0 2024-04-25T20:59:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 Processor -- start 2024-04-25T20:59:00.161 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 -- start start 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe756ffd640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe74c0a3fb0 0x7fe74c0a6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420324a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b42032f800 0x55b420331bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b420289970 con 0x55b420324620 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4202264a0 con 0x55b42032f800 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe76ae45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b420285930 con 0x55b420324f40 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe74c0a3fb0 0x7fe74c0a6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420324a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420324a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42768/0 (socket says 172.21.15.96:42768) 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe765ed2640 1 -- 172.21.15.96:0/745934041 learned_addr learned my addr 172.21.15.96:0/745934041 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:00.162 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe7656d1640 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.154+0000 7fe7666d3640 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b42032f800 0x55b420331bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe756ffd640 1 -- 172.21.15.96:0/3124569141 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe75c0f12f0 con 0x7fe7601010f0 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe765ed2640 1 -- 172.21.15.96:0/745934041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 msgr2=0x55b42032f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe765ed2640 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe765ed2640 1 -- 172.21.15.96:0/745934041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b42032f800 msgr2=0x55b420331bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe765ed2640 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b42032f800 0x55b420331bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe765ed2640 1 -- 172.21.15.96:0/745934041 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b41ff9c210 con 0x55b420324620 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe74c0a3fb0 0x7fe74c0a6470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe758009fd0 tx=0x7fe758009b60 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe765ed2640 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420324a00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0bd050 tx=0x7fe75c03b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d69871e00c2812 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe74bfff640 1 -- 172.21.15.96:0/745934041 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c049070 con 0x55b420324620 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe74bfff640 1 -- 172.21.15.96:0/745934041 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c0c6bf0 con 0x55b420324620 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.155+0000 7fe74bfff640 1 -- 172.21.15.96:0/745934041 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0bded0 con 0x55b420324620 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 -- 172.21.15.96:0/745934041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 msgr2=0x55b420324a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420324a00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0bd050 tx=0x7fe75c03b890 comp rx=0 tx=0).stop 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 -- 172.21.15.96:0/745934041 shutdown_connections 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b42032f800 0x55b420331bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 --2- 172.21.15.96:0/745934041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420324a00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 -- 172.21.15.96:0/745934041 >> 172.21.15.96:0/745934041 conn(0x55b420320b30 msgr2=0x55b420320f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 -- 172.21.15.96:0/745934041 shutdown_connections 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 -- 172.21.15.96:0/745934041 wait complete. 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 -- start start 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.156+0000 7fe76ae45740 1 Processor -- start 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42032f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b42032c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b4203321f0 con 0x55b42032f800 2024-04-25T20:59:00.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b4202240b0 con 0x55b420324620 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b420332370 con 0x55b420324f40 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe76ae45740 10 client.0 _ll_register_callbacks cb 0x55b420211d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b42032c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b42032c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42778/0 (socket says 172.21.15.96:42778) 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 -- 172.21.15.96:0/3881734781 learned_addr learned my addr 172.21.15.96:0/3881734781 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42032f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 -- 172.21.15.96:0/3881734781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 msgr2=0x55b42032c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 -- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 msgr2=0x55b42032f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42032f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 -- 172.21.15.96:0/3881734781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b420068620 con 0x55b42032f800 2024-04-25T20:59:00.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b42032c9b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe758011bc0 tx=0x7fe758004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e3b89c054e8d98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7497fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.157+0000 7fe7497fa640 1 -- 172.21.15.96:0/3881734781 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758023030 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe7497fa640 1 -- 172.21.15.96:0/3881734781 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe75800ebe0 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c0c6bf0 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe7497fa640 1 -- 172.21.15.96:0/3881734781 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe75802f050 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 10 client.15477 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b42033e0d0 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe7497fa640 1 -- 172.21.15.96:0/3881734781 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75c0c6bf0 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe7497fa640 1 -- 172.21.15.96:0/3881734781 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe758004550 con 0x55b42032f800 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 10 client.15477 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 20 client.15477 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:00.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b42033f590 0x55b4203419c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 4 client.15477 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b420297390 con 0x55b42033f590 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.158+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b42033f590 0x55b4203419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.159+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b42033f590 0x55b4203419c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b80290d100db995a server_cookie=98542d7d9b9ec692 in_seq=0 out_seq=0 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.159+0000 7fe7497fa640 10 client.15477 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe7497fa640 1 -- 172.21.15.96:0/3881734781 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b420297390 con 0x55b42033f590 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe7497fa640 10 client.15477 handle_command_reply: tid=0 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 client.15477 shutdown 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 msgr2=0x55b42032c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b42032c9b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe758011bc0 tx=0x7fe758004ab0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 shutdown_connections 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b42033f590 0x55b4203419c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b42032c9b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.167 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3881734781 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42032f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe7497fa640 0 client.15477 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 >> 172.21.15.96:0/3881734781 conn(0x55b420320b30 msgr2=0x55b4203268b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 shutdown_connections 2024-04-25T20:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 1 -- 172.21.15.96:0/3881734781 wait complete. 2024-04-25T20:59:00.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.160+0000 7fe76ae45740 20 client.15477 trim_cache size 0 max 16384 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.174+0000 7fe76ae45740 1 Processor -- start 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 -- start start 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42033afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b4203472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601b24c0 con 0x55b42032f800 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601afd90 con 0x55b420324620 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe76ae45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601adbe0 con 0x55b420324f40 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b4203472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b4203472d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42780/0 (socket says 172.21.15.96:42780) 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 -- 172.21.15.96:0/3449622766 learned_addr learned my addr 172.21.15.96:0/3449622766 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:00.180 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42033afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 -- 172.21.15.96:0/3449622766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 msgr2=0x55b42032ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 -- 172.21.15.96:0/3449622766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 msgr2=0x55b42033afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42033afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 -- 172.21.15.96:0/3449622766 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b420331bf0 con 0x55b42032f800 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42033afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b4203472d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe7580094d0 tx=0x7fe7580119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6873cec7744b8f8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe74b7fe640 1 -- 172.21.15.96:0/3449622766 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758038070 con 0x55b42032f800 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe74b7fe640 1 -- 172.21.15.96:0/3449622766 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75800ebe0 con 0x55b42032f800 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.175+0000 7fe74b7fe640 1 -- 172.21.15.96:0/3449622766 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758023040 con 0x55b42032f800 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 -- 172.21.15.96:0/3449622766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 msgr2=0x55b4203472d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.181 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b4203472d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fe7580094d0 tx=0x7fe7580119f0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 -- 172.21.15.96:0/3449622766 shutdown_connections 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b42032f800 0x55b4203472d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b420324f40 0x55b42032ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3449622766 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324620 0x55b42033afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 -- 172.21.15.96:0/3449622766 >> 172.21.15.96:0/3449622766 conn(0x55b420320b30 msgr2=0x55b420327af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 -- 172.21.15.96:0/3449622766 shutdown_connections 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 -- 172.21.15.96:0/3449622766 wait complete. 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 -- start start 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.176+0000 7fe76ae45740 1 Processor -- start 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420337610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324f40 0x55b420332aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b42032f800 0x55b420332fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.182 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ae250 con 0x55b420324620 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601af6c0 con 0x55b420324f40 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7601ae920 con 0x55b42032f800 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe76ae45740 10 client.0 _ll_register_callbacks cb 0x7fe758011aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420337610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420337610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42784/0 (socket says 172.21.15.96:42784) 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 -- 172.21.15.96:0/3290654529 learned_addr learned my addr 172.21.15.96:0/3290654529 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324f40 0x55b420332aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 -- 172.21.15.96:0/3290654529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b42032f800 msgr2=0x55b420332fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe7666d3640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b42032f800 0x55b420332fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b42032f800 0x55b420332fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 -- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324f40 msgr2=0x55b420332aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324f40 0x55b420332aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 -- 172.21.15.96:0/3290654529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4203335a0 con 0x55b420324620 2024-04-25T20:59:00.183 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324f40 0x55b420332aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe765ed2640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420337610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0bb370 tx=0x7fe75c031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3172c9a4085f756 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.177+0000 7fe7497fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe7497fa640 1 -- 172.21.15.96:0/3290654529 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0db060 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe7497fa640 1 -- 172.21.15.96:0/3290654529 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe75c0c2840 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75800ebe0 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe7497fa640 1 -- 172.21.15.96:0/3290654529 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe75c049040 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 10 client.15483 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b4203a5870 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe7497fa640 1 -- 172.21.15.96:0/3290654529 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75800ebe0 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe7497fa640 1 -- 172.21.15.96:0/3290654529 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe75c0b9730 con 0x55b420324620 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 10 client.15483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:59:00.184 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 20 client.15483 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b4203a5de0 0x55b4203a8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 4 client.15483 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:59:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b420291200 con 0x55b4203a5de0 2024-04-25T20:59:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.178+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b4203a5de0 0x55b4203a8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.179+0000 7fe7497fa640 10 client.15483 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:00.185 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.179+0000 7fe7656d1640 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b4203a5de0 0x55b4203a8200 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=996262dcaf0a13c3 server_cookie=c83df68097106088 in_seq=0 out_seq=0 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.182+0000 7fe7497fa640 1 -- 172.21.15.96:0/3290654529 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b420291200 con 0x55b4203a5de0 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.182+0000 7fe7497fa640 10 client.15483 handle_command_reply: tid=0 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.182+0000 7fe76ae45740 1 client.15483 shutdown 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.182+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 msgr2=0x55b420337610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.182+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420337610 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0bb370 tx=0x7fe75c031f10 comp rx=0 tx=0).stop 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 shutdown_connections 2024-04-25T20:59:00.186 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b4203a5de0 0x55b4203a8200 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b42032f800 0x55b420332fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b420324f40 0x55b420332aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe7497fa640 0 client.15483 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 --2- 172.21.15.96:0/3290654529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b420324620 0x55b420337610 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 >> 172.21.15.96:0/3290654529 conn(0x55b420320b30 msgr2=0x55b420339fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 shutdown_connections 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 1 -- 172.21.15.96:0/3290654529 wait complete. 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe76ae45740 20 client.15483 trim_cache size 0 max 16384 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T20:59:00.187 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T20:59:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T20:59:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe74c0a3fb0 msgr2=0x7fe74c0a6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe74c0a3fb0 0x7fe74c0a6470 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe758009fd0 tx=0x7fe758009b60 comp rx=0 tx=0).stop 2024-04-25T20:59:00.188 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 msgr2=0x7fe760102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe760102b30 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe75c005ab0 tx=0x7fe75c003c80 comp rx=0 tx=0).stop 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 shutdown_connections 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe74c0a3fb0 0x7fe74c0a6470 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7601035b0 0x7fe7601a6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe760106170 0x7fe760103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 --2- 172.21.15.96:0/3124569141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe7601010f0 0x7fe760102b30 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 >> 172.21.15.96:0/3124569141 conn(0x7fe760077fd0 msgr2=0x7fe760104d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 shutdown_connections 2024-04-25T20:59:00.189 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:00.183+0000 7fe768136640 1 -- 172.21.15.96:0/3124569141 wait complete. 2024-04-25T20:59:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:00 smithi096 ceph-mon[29288]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T20:59:00.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:00 smithi097 ceph-mon[32667]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T20:59:01.203 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:59:01.203 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:00 smithi139 ceph-mon[33027]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-25T20:59:01.448 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:02 smithi096 ceph-mon[29288]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:59:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:02 smithi097 ceph-mon[32667]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:59:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:02 smithi139 ceph-mon[33027]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 -- 172.21.15.96:0/210286557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff774108a80 msgr2=0x7ff774108e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 --2- 172.21.15.96:0/210286557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff774108a80 0x7ff774108e60 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7ff764009990 tx=0x7ff764009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 -- 172.21.15.96:0/210286557 shutdown_connections 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 --2- 172.21.15.96:0/210286557 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff7741034b0 0x7ff77410f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 --2- 172.21.15.96:0/210286557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff774108a80 0x7ff774108e60 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 --2- 172.21.15.96:0/210286557 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff774102af0 0x7ff774102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 -- 172.21.15.96:0/210286557 >> 172.21.15.96:0/210286557 conn(0x7ff774079ad0 msgr2=0x7ff774079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:03.220 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 -- 172.21.15.96:0/210286557 shutdown_connections 2024-04-25T20:59:03.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.219+0000 7ff779081700 1 -- 172.21.15.96:0/210286557 wait complete. 2024-04-25T20:59:03.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 Processor -- start 2024-04-25T20:59:03.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 -- start start 2024-04-25T20:59:03.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff774102af0 0x7ff774199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:03.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 0x7ff774199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:03.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff774108a80 0x7ff77419d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff774112c30 con 0x7ff7741034b0 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff774112ab0 con 0x7ff774102af0 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff779081700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff774112db0 con 0x7ff774108a80 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 0x7ff774199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff772d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff774102af0 0x7ff774199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 0x7ff774199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42796/0 (socket says 172.21.15.96:42796) 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 -- 172.21.15.96:0/662361258 learned_addr learned my addr 172.21.15.96:0/662361258 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff77359e700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff774108a80 0x7ff77419d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 -- 172.21.15.96:0/662361258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff774108a80 msgr2=0x7ff77419d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff774108a80 0x7ff77419d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 -- 172.21.15.96:0/662361258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff774102af0 msgr2=0x7ff774199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff774102af0 0x7ff774199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.220+0000 7ff76a59c700 1 -- 172.21.15.96:0/662361258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77419e010 con 0x7ff7741034b0 2024-04-25T20:59:03.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.221+0000 7ff76a59c700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 0x7ff774199540 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c9b0 tx=0x7ff75c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.221+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c007b00 con 0x7ff7741034b0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.221+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77419e300 con 0x7ff7741034b0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.221+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff75c00ce50 con 0x7ff7741034b0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.221+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c018660 con 0x7ff7741034b0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.221+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7741a5b80 con 0x7ff7741034b0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.222+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff774069730 con 0x7ff7741034b0 2024-04-25T20:59:03.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.224+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7ff75c00f450 con 0x7ff7741034b0 2024-04-25T20:59:03.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.224+0000 7ff770d99700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff7600a3f60 0x7ff7600a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:03.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.224+0000 7ff772d9d700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff7600a3f60 0x7ff7600a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:03.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.224+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7ff75c0d0c20 con 0x7ff7741034b0 2024-04-25T20:59:03.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.225+0000 7ff772d9d700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff7600a3f60 0x7ff7600a6420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff76400b5c0 tx=0x7ff764005960 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:03.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.228+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff75c014030 con 0x7ff7741034b0 2024-04-25T20:59:03.471 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.470+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff77402df80 con 0x7ff7741034b0 2024-04-25T20:59:03.472 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.471+0000 7ff770d99700 1 -- 172.21.15.96:0/662361258 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11859 (secure 0 0 0) 0x7ff75c09d180 con 0x7ff7741034b0 2024-04-25T20:59:03.472 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:59:03.472 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:57:37.187405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T20:59:03.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff7600a3f60 msgr2=0x7ff7600a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:03.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff7600a3f60 0x7ff7600a6420 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff76400b5c0 tx=0x7ff764005960 comp rx=0 tx=0).stop 2024-04-25T20:59:03.474 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 msgr2=0x7ff774199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 0x7ff774199540 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c9b0 tx=0x7ff75c00ccc0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 shutdown_connections 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff7600a3f60 0x7ff7600a6420 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff774108a80 0x7ff77419d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff774102af0 0x7ff774199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 --2- 172.21.15.96:0/662361258 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7741034b0 0x7ff774199540 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 >> 172.21.15.96:0/662361258 conn(0x7ff774079ad0 msgr2=0x7ff774101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 shutdown_connections 2024-04-25T20:59:03.475 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:03.473+0000 7ff779081700 1 -- 172.21.15.96:0/662361258 wait complete. 2024-04-25T20:59:03.476 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T20:59:04.217 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24443 quiesce db --set-id fd1653f2 --release --await' 2024-04-25T20:59:04.217 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24443 quiesce db --set-id fd1653f2 --release --await 2024-04-25T20:59:04.454 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:04 smithi096 ceph-mon[29288]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:04.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:04 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/662361258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:04 smithi097 ceph-mon[32667]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:04 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/662361258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:04 smithi139 ceph-mon[33027]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T20:59:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:04 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/662361258' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 -- 172.21.15.96:0/1059335361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54101400 msgr2=0x7f4c54101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 --2- 172.21.15.96:0/1059335361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54101400 0x7f4c54101880 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009990 tx=0x7f4c3c009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 -- 172.21.15.96:0/1059335361 shutdown_connections 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 --2- 172.21.15.96:0/1059335361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54101dc0 0x7f4c5410f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 --2- 172.21.15.96:0/1059335361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54107b20 0x7f4c54107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 --2- 172.21.15.96:0/1059335361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54101400 0x7f4c54101880 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 -- 172.21.15.96:0/1059335361 >> 172.21.15.96:0/1059335361 conn(0x7f4c54076c60 msgr2=0x7f4c54077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 -- 172.21.15.96:0/1059335361 shutdown_connections 2024-04-25T20:59:06.148 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.147+0000 7f4c5917f700 1 -- 172.21.15.96:0/1059335361 wait complete. 2024-04-25T20:59:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 Processor -- start 2024-04-25T20:59:06.149 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 -- start start 2024-04-25T20:59:06.150 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54101400 0x7f4c541991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54101dc0 0x7f4c54199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 0x7f4c5419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54112cb0 con 0x7f4c54107b20 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54112b30 con 0x7f4c54101dc0 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5917f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c54112e30 con 0x7f4c54101400 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 0x7f4c5419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5359e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 0x7f4c5419da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42828/0 (socket says 172.21.15.96:42828) 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c5359e700 1 -- 172.21.15.96:0/2062763140 learned_addr learned my addr 172.21.15.96:0/2062763140 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54101dc0 0x7f4c54199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.148+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54101400 0x7f4c541991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5359e700 1 -- 172.21.15.96:0/2062763140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54101400 msgr2=0x7f4c541991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5359e700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54101400 0x7f4c541991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.151 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5359e700 1 -- 172.21.15.96:0/2062763140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54101dc0 msgr2=0x7f4c54199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5359e700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54101dc0 0x7f4c54199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5359e700 1 -- 172.21.15.96:0/2062763140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5419e1a0 con 0x7f4c54107b20 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54101400 0x7f4c541991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5359e700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 0x7f4c5419da60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c007ee0 tx=0x7f4c4c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c50d99700 1 -- 172.21.15.96:0/2062763140 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00dbd0 con 0x7f4c54107b20 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c50d99700 1 -- 172.21.15.96:0/2062763140 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c4c00dd70 con 0x7f4c54107b20 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c50d99700 1 -- 172.21.15.96:0/2062763140 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c005990 con 0x7f4c54107b20 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5419e490 con 0x7f4c54107b20 2024-04-25T20:59:06.152 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.149+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c541a5d10 con 0x7f4c54107b20 2024-04-25T20:59:06.153 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.152+0000 7f4c50d99700 1 -- 172.21.15.96:0/2062763140 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7f4c4c01a020 con 0x7f4c54107b20 2024-04-25T20:59:06.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.152+0000 7f4c50d99700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4c400a3f60 0x7f4c400a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.152+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4c400a3f60 0x7f4c400a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.152+0000 7f4c50d99700 1 -- 172.21.15.96:0/2062763140 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f4c4c0d0230 con 0x7f4c54107b20 2024-04-25T20:59:06.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 Processor -- start 2024-04-25T20:59:06.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 -- start start 2024-04-25T20:59:06.156 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4c400a3f60 0x7f4c400a6420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c540781e0 tx=0x7f4c44008040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 0x55b068c2ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c39930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c39e70 0x55b068c3c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b068b81d90 con 0x55b068c2e650 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b068b28a30 con 0x55b068c2ef70 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b068b7dd50 con 0x55b068c39e70 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c52d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 0x55b068c2ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c52d9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 0x55b068c2ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42840/0 (socket says 172.21.15.96:42840) 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c52d9d700 1 -- 172.21.15.96:0/3385137512 learned_addr learned my addr 172.21.15.96:0/3385137512 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c5359e700 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c39e70 0x55b068c3c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.153+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c39930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c52d9d700 1 -- 172.21.15.96:0/3385137512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c39e70 msgr2=0x55b068c3c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.158 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c39e70 0x55b068c3c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c52d9d700 1 -- 172.21.15.96:0/3385137512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 msgr2=0x55b068c39930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c39930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c52d9d700 1 -- 172.21.15.96:0/3385137512 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b068aec840 con 0x55b068c2e650 2024-04-25T20:59:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c5359e700 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c39e70 0x55b068c3c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.154+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 0x55b068c2ea30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400d7a0 tx=0x7f4c44010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40b42e26e7e54e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:06.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.155+0000 7f4c4959a700 1 -- 172.21.15.96:0/3385137512 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c44024070 con 0x55b068c2e650 2024-04-25T20:59:06.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.155+0000 7f4c4959a700 1 -- 172.21.15.96:0/3385137512 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c44014af0 con 0x55b068c2e650 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.155+0000 7f4c4959a700 1 -- 172.21.15.96:0/3385137512 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4401fbd0 con 0x55b068c2e650 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 -- 172.21.15.96:0/3385137512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 msgr2=0x55b068c2ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 0x55b068c2ea30 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4c4400d7a0 tx=0x7f4c44010000 comp rx=0 tx=0).stop 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 -- 172.21.15.96:0/3385137512 shutdown_connections 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c39e70 0x55b068c3c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2e650 0x55b068c2ea30 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/3385137512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c39930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 -- 172.21.15.96:0/3385137512 >> 172.21.15.96:0/3385137512 conn(0x55b068b5b990 msgr2=0x55b068c2ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 -- 172.21.15.96:0/3385137512 shutdown_connections 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 -- 172.21.15.96:0/3385137512 wait complete. 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.156+0000 7f4c63a08b80 1 -- start start 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 Processor -- start 2024-04-25T20:59:06.163 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c395f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c368f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c39e70 0x55b068c36e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b068c40d10 con 0x55b068c39e70 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55b068b3b6b0 con 0x55b068c2ef70 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55b068c40e90 con 0x55b068c2e650 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c63a08b80 10 client.0 _ll_register_callbacks cb 0x55b068b1aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c368f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c368f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:39194/0 (socket says 172.21.15.96:39194) 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c4a59c700 1 -- 172.21.15.96:0/300681874 learned_addr learned my addr 172.21.15.96:0/300681874 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c39e70 0x55b068c36e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c395f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 -- 172.21.15.96:0/300681874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 msgr2=0x55b068c395f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c395f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.164 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 -- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 msgr2=0x55b068c368f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c368f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 -- 172.21.15.96:0/300681874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b068acf9d0 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c5359e700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c39e70 0x55b068c36e30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0cd7f0 tx=0x7f4c4c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f062cfbc0ebe45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.157+0000 7f4c3a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/300681874 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c0a0320 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/300681874 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f4c4c09b6c0 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c44014af0 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/300681874 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c4c09b8c0 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 10 client.15501 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b068c4b620 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/300681874 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c44014af0 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/300681874 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4c4c0b0c00 con 0x55b068c39e70 2024-04-25T20:59:06.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 10 client.15501 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 10 client.15501 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 20 client.15501 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068c500a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 4 client.15501 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b068b8f580 con 0x55b068c4dc60 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.158+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068c500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.159+0000 7f4c3a7fc700 10 client.15501 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.159+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068c500a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c1acd0456bb246 server_cookie=7ee3f9085657205c in_seq=0 out_seq=0 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/300681874 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b068b8f580 con 0x55b068c4dc60 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c3a7fc700 10 client.15501 handle_command_reply: tid=0 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 client.15501 shutdown 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c39e70 msgr2=0x55b068c36e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c39e70 0x55b068c36e30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c0cd7f0 tx=0x7f4c4c0192c0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.166 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 shutdown_connections 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c39e70 0x55b068c36e30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068c500a0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c3a7fc700 0 client.15501 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c395f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/300681874 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c2ef70 0x55b068c368f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 >> 172.21.15.96:0/300681874 conn(0x55b068b5b990 msgr2=0x55b068c30ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 shutdown_connections 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 1 -- 172.21.15.96:0/300681874 wait complete. 2024-04-25T20:59:06.167 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.160+0000 7f4c63a08b80 20 client.15501 trim_cache size 0 max 16384 2024-04-25T20:59:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.179+0000 7f4c63a08b80 1 Processor -- start 2024-04-25T20:59:06.180 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.179+0000 7f4c63a08b80 1 -- start start 2024-04-25T20:59:06.180 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:06 smithi096 ceph-mon[29288]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T20:59:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c49da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c55970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ad8e0 con 0x55b068c2ef70 2024-04-25T20:59:06.181 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541ab170 con 0x55b068c39e70 2024-04-25T20:59:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a9000 con 0x55b068c2e650 2024-04-25T20:59:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42852/0 (socket says 172.21.15.96:42852) 2024-04-25T20:59:06.182 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 -- 172.21.15.96:0/1510758783 learned_addr learned my addr 172.21.15.96:0/1510758783 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c49da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c5359e700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c55970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 -- 172.21.15.96:0/1510758783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 msgr2=0x55b068c49da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c49da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.183 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 -- 172.21.15.96:0/1510758783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 msgr2=0x55b068c55970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c55970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 -- 172.21.15.96:0/1510758783 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b068c34ea0 con 0x55b068c2ef70 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c5359e700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c55970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.180+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c390c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0059d0 tx=0x7f4c3c005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b4d378147ba6220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c48d99700 1 -- 172.21.15.96:0/1510758783 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c027020 con 0x55b068c2ef70 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c48d99700 1 -- 172.21.15.96:0/1510758783 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3c003860 con 0x55b068c2ef70 2024-04-25T20:59:06.184 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c48d99700 1 -- 172.21.15.96:0/1510758783 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c0054b0 con 0x55b068c2ef70 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 -- 172.21.15.96:0/1510758783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 msgr2=0x55b068c390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c390c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c0059d0 tx=0x7f4c3c005960 comp rx=0 tx=0).stop 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 -- 172.21.15.96:0/1510758783 shutdown_connections 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c55970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c49da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/1510758783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c390c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 -- 172.21.15.96:0/1510758783 >> 172.21.15.96:0/1510758783 conn(0x55b068b5b990 msgr2=0x55b068c389b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 -- 172.21.15.96:0/1510758783 shutdown_connections 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.181+0000 7f4c63a08b80 1 -- 172.21.15.96:0/1510758783 wait complete. 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 -- start start 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 Processor -- start 2024-04-25T20:59:06.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c46310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c42be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a9650 con 0x55b068c2ef70 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541aaaa0 con 0x55b068c39e70 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4c541a9d00 con 0x55b068c2e650 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c63a08b80 10 client.0 _ll_register_callbacks cb 0x55b068b1aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c4a59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:42866/0 (socket says 172.21.15.96:42866) 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c4a59c700 1 -- 172.21.15.96:0/878786341 learned_addr learned my addr 172.21.15.96:0/878786341 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c5359e700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c42be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.182+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c46310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c4a59c700 1 -- 172.21.15.96:0/878786341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 msgr2=0x55b068c46310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c46310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c4a59c700 1 -- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 msgr2=0x55b068c42be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c42be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c4a59c700 1 -- 172.21.15.96:0/878786341 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b068c43220 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c4a59c700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c426a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009990 tx=0x7f4c3c00bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=727d8cff577651e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c3a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/878786341 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c3c005520 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c3c003860 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/878786341 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f4c3c00f460 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/878786341 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4c3c03e050 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c63a08b80 10 client.15507 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55b068cb5010 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.183+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/878786341 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c3c003860 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/878786341 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f4c3c02c420 con 0x55b068c2ef70 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c63a08b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:06.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c63a08b80 10 client.15507 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c63a08b80 20 client.15507 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068cb7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c63a08b80 4 client.15507 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fd1653f2", "release": true, "await": true}] 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fd1653f2", "release": true, "await": true}) -- 0x55b068b89620 con 0x55b068c4dc60 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068cb7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c3a7fc700 10 client.15507 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:06.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.184+0000 7f4c52d9d700 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068cb7670 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80abf228aea108bc server_cookie=14f8e6316dc149d in_seq=0 out_seq=0 2024-04-25T20:59:06.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c3a7fc700 1 -- 172.21.15.96:0/878786341 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55b068b89620 con 0x55b068c4dc60 2024-04-25T20:59:06.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c3a7fc700 10 client.15507 handle_command_reply: tid=0 2024-04-25T20:59:06.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 client.15507 shutdown 2024-04-25T20:59:06.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 msgr2=0x55b068c426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.191 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c426a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f4c3c009990 tx=0x7f4c3c00bc40 comp rx=0 tx=0).stop 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 shutdown_connections 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b068c39e70 0x55b068c42be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55b068c4dc60 0x55b068cb7670 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55b068c2e650 0x55b068c46310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 --2- 172.21.15.96:0/878786341 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b068c2ef70 0x55b068c426a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 >> 172.21.15.96:0/878786341 conn(0x55b068b5b990 msgr2=0x55b068c390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c3a7fc700 0 client.15507 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 shutdown_connections 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 1 -- 172.21.15.96:0/878786341 wait complete. 2024-04-25T20:59:06.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.188+0000 7f4c63a08b80 20 client.15507 trim_cache size 0 max 16384 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 26, 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "leader": 24443, 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "set_version": 4, 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "sets": { 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "fd1653f2": { 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "version": 4, 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age_ref": 0.0, 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RELEASED", 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0 2024-04-25T20:59:06.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "timeout": 60.0, 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "expiration": 86.2, 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "members": { 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "file:/": { 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "excluded": false, 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RELEASED", 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:06.194 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:59:06.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4c400a3f60 msgr2=0x7f4c400a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4c400a3f60 0x7f4c400a6420 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c540781e0 tx=0x7f4c44008040 comp rx=0 tx=0).stop 2024-04-25T20:59:06.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 msgr2=0x7f4c5419da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:06.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 0x7f4c5419da60 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c007ee0 tx=0x7f4c4c00d3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 shutdown_connections 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4c400a3f60 0x7f4c400a6420 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4c54107b20 0x7f4c5419da60 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4c54101400 0x7f4c541991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 --2- 172.21.15.96:0/2062763140 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4c54101dc0 0x7f4c54199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 >> 172.21.15.96:0/2062763140 conn(0x7f4c54076c60 msgr2=0x7f4c540fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 shutdown_connections 2024-04-25T20:59:06.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:06.189+0000 7f4c5917f700 1 -- 172.21.15.96:0/2062763140 wait complete. 2024-04-25T20:59:06.196 INFO:tasks.workunit.client.0.smithi096.stderr:+ set -e 2024-04-25T20:59:06.197 INFO:tasks.workunit.client.0.smithi096.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-04-25T20:59:06.197 INFO:tasks.workunit.client.0.smithi096.stderr:Cloning into 'xfstests-dev'... 2024-04-25T20:59:06.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:06 smithi097 ceph-mon[32667]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T20:59:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:06 smithi139 ceph-mon[33027]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 20 op/s 2024-04-25T20:59:07.006 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fd1653f2', seconds elapsed: 39.1 2024-04-25T20:59:07.006 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.2 2024-04-25T20:59:07.006 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:59:07.249 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:08.446 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:08 smithi096 ceph-mon[29288]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T20:59:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:08 smithi097 ceph-mon[32667]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T20:59:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:08 smithi139 ceph-mon[33027]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T20:59:08.946 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 -- 172.21.15.96:0/40447515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 msgr2=0x7fe61406abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 --2- 172.21.15.96:0/40447515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe61406abb0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe604009990 tx=0x7fe604009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 -- 172.21.15.96:0/40447515 shutdown_connections 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 --2- 172.21.15.96:0/40447515 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61406b0f0 0x7fe61410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 --2- 172.21.15.96:0/40447515 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe614108a30 0x7fe614108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 --2- 172.21.15.96:0/40447515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe61406abb0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 -- 172.21.15.96:0/40447515 >> 172.21.15.96:0/40447515 conn(0x7fe6140788f0 msgr2=0x7fe614078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 -- 172.21.15.96:0/40447515 shutdown_connections 2024-04-25T20:59:08.947 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.945+0000 7fe61b76b700 1 -- 172.21.15.96:0/40447515 wait complete. 2024-04-25T20:59:08.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.946+0000 7fe61b76b700 1 Processor -- start 2024-04-25T20:59:08.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.946+0000 7fe61b76b700 1 -- start start 2024-04-25T20:59:08.949 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.946+0000 7fe61b76b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe614199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.946+0000 7fe61b76b700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61406b0f0 0x7fe614199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.946+0000 7fe61b76b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe614108a30 0x7fe61419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe61b76b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe614112c30 con 0x7fe61406a750 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe61b76b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe614112ab0 con 0x7fe614108a30 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe61b76b700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe614112db0 con 0x7fe61406b0f0 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe614199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe614199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45514/0 (socket says 172.21.15.96:45514) 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 -- 172.21.15.96:0/2345809767 learned_addr learned my addr 172.21.15.96:0/2345809767 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619d08700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe614108a30 0x7fe61419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe618d06700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61406b0f0 0x7fe614199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 -- 172.21.15.96:0/2345809767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61406b0f0 msgr2=0x7fe614199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:08.950 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61406b0f0 0x7fe614199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 -- 172.21.15.96:0/2345809767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe614108a30 msgr2=0x7fe61419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe614108a30 0x7fe61419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 -- 172.21.15.96:0/2345809767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61419e0e0 con 0x7fe61406a750 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe619507700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe614199120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe61000d980 tx=0x7fe61000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe610016070 con 0x7fe61406a750 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61419e3d0 con 0x7fe61406a750 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe610004830 con 0x7fe61406a750 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6100054c0 con 0x7fe61406a750 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.947+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6141a5c50 con 0x7fe61406a750 2024-04-25T20:59:08.951 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.948+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe614069730 con 0x7fe61406a750 2024-04-25T20:59:08.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.950+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138930+0+0 (secure 0 0 0) 0x7fe610004220 con 0x7fe61406a750 2024-04-25T20:59:08.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.950+0000 7fe60a7fc700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe6000a3f60 0x7fe6000a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:08.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.950+0000 7fe618d06700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe6000a3f60 0x7fe6000a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:08.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.950+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe6100d07f0 con 0x7fe61406a750 2024-04-25T20:59:08.952 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.951+0000 7fe618d06700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe6000a3f60 0x7fe6000a6420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe604006010 tx=0x7fe60401a040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:08.956 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:08.955+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe61001f020 con 0x7fe61406a750 2024-04-25T20:59:09.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.200+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe61402df80 con 0x7fe61406a750 2024-04-25T20:59:09.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.201+0000 7fe60a7fc700 1 -- 172.21.15.96:0/2345809767 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11859 (secure 0 0 0) 0x7fe61009ccc0 con 0x7fe61406a750 2024-04-25T20:59:09.202 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:59:09.202 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:57:37.187405+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T20:59:09.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe6000a3f60 msgr2=0x7fe6000a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:09.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe6000a3f60 0x7fe6000a6420 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe604006010 tx=0x7fe60401a040 comp rx=0 tx=0).stop 2024-04-25T20:59:09.204 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 msgr2=0x7fe614199120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe614199120 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe61000d980 tx=0x7fe61000dc90 comp rx=0 tx=0).stop 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 shutdown_connections 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe6000a3f60 0x7fe6000a6420 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe614108a30 0x7fe61419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe61406a750 0x7fe614199120 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 --2- 172.21.15.96:0/2345809767 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe61406b0f0 0x7fe614199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.203+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 >> 172.21.15.96:0/2345809767 conn(0x7fe6140788f0 msgr2=0x7fe6140ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.204+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 shutdown_connections 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:09.204+0000 7fe61b76b700 1 -- 172.21.15.96:0/2345809767 wait complete. 2024-04-25T20:59:09.205 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 26 2024-04-25T20:59:09.918 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24443 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0' 2024-04-25T20:59:09.918 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24443 quiesce db / --timeout 60 --expiration 86.2 --await-for 42.0 2024-04-25T20:59:10.155 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:10.430 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:10 smithi096 ceph-mon[29288]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T20:59:10.430 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:10 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2345809767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:10 smithi097 ceph-mon[32667]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T20:59:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:10 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2345809767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:10 smithi139 ceph-mon[33027]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 386 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T20:59:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:10 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2345809767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 -- 172.21.15.96:0/2957815060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006b990 msgr2=0x7f608010f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 --2- 172.21.15.96:0/2957815060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006b990 0x7f608010f920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f607800b0a0 tx=0x7f607800b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 -- 172.21.15.96:0/2957815060 shutdown_connections 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 --2- 172.21.15.96:0/2957815060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006b990 0x7f608010f920 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 --2- 172.21.15.96:0/2957815060 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080108a50 0x7f6080108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 --2- 172.21.15.96:0/2957815060 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006aff0 0x7f608006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 -- 172.21.15.96:0/2957815060 >> 172.21.15.96:0/2957815060 conn(0x7f6080078980 msgr2=0x7f6080078d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:11.779 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 -- 172.21.15.96:0/2957815060 shutdown_connections 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.776+0000 7f6084e95700 1 -- 172.21.15.96:0/2957815060 wait complete. 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 Processor -- start 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 -- start start 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006aff0 0x7f6080198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006b990 0x7f60801994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080108a50 0x7f608019d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6080112bf0 con 0x7f608006aff0 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6080112a70 con 0x7f6080108a50 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f6084e95700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6080112d70 con 0x7f608006b990 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f607e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006aff0 0x7f6080198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f607e59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006aff0 0x7f6080198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45522/0 (socket says 172.21.15.96:45522) 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f607e59c700 1 -- 172.21.15.96:0/2298970808 learned_addr learned my addr 172.21.15.96:0/2298970808 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.777+0000 7f607ed9d700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080108a50 0x7f608019d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.780 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006b990 0x7f60801994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607e59c700 1 -- 172.21.15.96:0/2298970808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006b990 msgr2=0x7f60801994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607e59c700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f608006b990 0x7f60801994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607e59c700 1 -- 172.21.15.96:0/2298970808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080108a50 msgr2=0x7f608019d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607e59c700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6080108a50 0x7f608019d870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607e59c700 1 -- 172.21.15.96:0/2298970808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608019dfb0 con 0x7f608006aff0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f607e59c700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f608006aff0 0x7f6080198fa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f607000d950 tx=0x7f607000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f606f7fe700 1 -- 172.21.15.96:0/2298970808 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070016070 con 0x7f608006aff0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f6084e95700 1 -- 172.21.15.96:0/2298970808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608019e2a0 con 0x7f608006aff0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f606f7fe700 1 -- 172.21.15.96:0/2298970808 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6070009df0 con 0x7f608006aff0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f606f7fe700 1 -- 172.21.15.96:0/2298970808 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607000f3c0 con 0x7f608006aff0 2024-04-25T20:59:11.781 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.778+0000 7f6084e95700 1 -- 172.21.15.96:0/2298970808 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60801a5b20 con 0x7f608006aff0 2024-04-25T20:59:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.780+0000 7f606f7fe700 1 -- 172.21.15.96:0/2298970808 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6070019070 con 0x7f608006aff0 2024-04-25T20:59:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.781+0000 7f606f7fe700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f60680a3e50 0x7f60680a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.781+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f60680a3e50 0x7f60680a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.781+0000 7f606f7fe700 1 -- 172.21.15.96:0/2298970808 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f60700d0300 con 0x7f608006aff0 2024-04-25T20:59:11.782 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.781+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2298970808 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f60680a3e50 0x7f60680a6310 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6074005950 tx=0x7f60740058e0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:11.784 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 Processor -- start 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 -- start start 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5db9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ddb523e20 con 0x562ddb5d1000 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562ddb4caac0 con 0x562ddb5dbf00 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f608f71eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562ddb51fde0 con 0x562ddb5d06e0 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f607dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5db9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f607dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5db9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45538/0 (socket says 172.21.15.96:45538) 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f607dd9b700 1 -- 172.21.15.96:0/2172244344 learned_addr learned my addr 172.21.15.96:0/2172244344 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f607ed9d700 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.782+0000 7f607e59c700 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.785 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607dd9b700 1 -- 172.21.15.96:0/2172244344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 msgr2=0x562ddb5d0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607dd9b700 1 -- 172.21.15.96:0/2172244344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 msgr2=0x562ddb5de2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5de2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607dd9b700 1 -- 172.21.15.96:0/2172244344 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ddb48e8d0 con 0x562ddb5d1000 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607ed9d700 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5de2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5db9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f607400fd40 tx=0x7f6074012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87de39bafdc9b7f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f606cff9700 1 -- 172.21.15.96:0/2172244344 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074022070 con 0x562ddb5d1000 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f606cff9700 1 -- 172.21.15.96:0/2172244344 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6074011bb0 con 0x562ddb5d1000 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.783+0000 7f606cff9700 1 -- 172.21.15.96:0/2172244344 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607401d610 con 0x562ddb5d1000 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 -- 172.21.15.96:0/2172244344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 msgr2=0x562ddb5db9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5db9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f607400fd40 tx=0x7f6074012000 comp rx=0 tx=0).stop 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 -- 172.21.15.96:0/2172244344 shutdown_connections 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5de2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.786 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2172244344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5db9c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 -- 172.21.15.96:0/2172244344 >> 172.21.15.96:0/2172244344 conn(0x562ddb4fda20 msgr2=0x562ddb5ccba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 -- 172.21.15.96:0/2172244344 shutdown_connections 2024-04-25T20:59:11.787 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 -- 172.21.15.96:0/2172244344 wait complete. 2024-04-25T20:59:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 -- start start 2024-04-25T20:59:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.784+0000 7f608f71eb80 1 Processor -- start 2024-04-25T20:59:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5db6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.790 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d1000 0x562ddb5d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ddb5de990 con 0x562ddb5dbf00 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562ddb4dd740 con 0x562ddb5d1000 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562ddb5deb10 con 0x562ddb5d06e0 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 10 client.0 _ll_register_callbacks cb 0x562ddb4bcab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5d8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45540/0 (socket says 172.21.15.96:45540) 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 -- 172.21.15.96:0/2507493456 learned_addr learned my addr 172.21.15.96:0/2507493456 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607e59c700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5db6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d1000 0x562ddb5d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 -- 172.21.15.96:0/2507493456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 msgr2=0x562ddb5db6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5db6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 -- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d1000 msgr2=0x562ddb5d89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.791 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d1000 0x562ddb5d89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 -- 172.21.15.96:0/2507493456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ddb471a60 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607dd9b700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d1000 0x562ddb5d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f607ed9d700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5d8f10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6078004c70 tx=0x7f6078004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c6fdf27f79cdc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f6065ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f6065ffb700 1 -- 172.21.15.96:0/2507493456 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60780078f0 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f6065ffb700 1 -- 172.21.15.96:0/2507493456 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6078007a90 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.785+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6074011bb0 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f6065ffb700 1 -- 172.21.15.96:0/2507493456 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6078007c90 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 10 client.15525 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ddb5e9290 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f6065ffb700 1 -- 172.21.15.96:0/2507493456 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6074011bb0 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f6065ffb700 1 -- 172.21.15.96:0/2507493456 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6078030050 con 0x562ddb5dbf00 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 10 client.15525 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 10 client.15525 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T20:59:11.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 20 client.15525 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb5edc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 4 client.15525 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ddb531610 con 0x562ddb5eb820 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.786+0000 7f607e59c700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb5edc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.787+0000 7f6065ffb700 10 client.15525 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.787+0000 7f607e59c700 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb5edc40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5194f018cc20602 server_cookie=31d180a41b63d1ce in_seq=0 out_seq=0 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.787+0000 7f6065ffb700 1 -- 172.21.15.96:0/2507493456 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562ddb531610 con 0x562ddb5eb820 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.787+0000 7f6065ffb700 10 client.15525 handle_command_reply: tid=0 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.787+0000 7f608f71eb80 1 client.15525 shutdown 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 msgr2=0x562ddb5d8f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5d8f10 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f6078004c70 tx=0x7f6078004d50 comp rx=0 tx=0).stop 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 shutdown_connections 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5d8f10 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb5edc40 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5db6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f6065ffb700 0 client.15525 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 --2- 172.21.15.96:0/2507493456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d1000 0x562ddb5d89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 >> 172.21.15.96:0/2507493456 conn(0x562ddb4fda20 msgr2=0x562ddb5d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 shutdown_connections 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 1 -- 172.21.15.96:0/2507493456 wait complete. 2024-04-25T20:59:11.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.788+0000 7f608f71eb80 20 client.15525 trim_cache size 0 max 16384 2024-04-25T20:59:11.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 Processor -- start 2024-04-25T20:59:11.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 -- start start 2024-04-25T20:59:11.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.808 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5e7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5f3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60801af8e0 con 0x562ddb5d1000 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60801ad170 con 0x562ddb5dbf00 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f608f71eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60801aafc0 con 0x562ddb5d06e0 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5e7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5e7890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45546/0 (socket says 172.21.15.96:45546) 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 -- 172.21.15.96:0/3030440519 learned_addr learned my addr 172.21.15.96:0/3030440519 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 -- 172.21.15.96:0/3030440519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 msgr2=0x562ddb5d9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 -- 172.21.15.96:0/3030440519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 msgr2=0x562ddb5f3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 -- 172.21.15.96:0/3030440519 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ddb5d6f60 con 0x562ddb5d1000 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f607dd9b700 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5e7890 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f607400ff70 tx=0x7f607400ffa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31003919c7498514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.807+0000 7f6067fff700 1 -- 172.21.15.96:0/3030440519 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6074022040 con 0x562ddb5d1000 2024-04-25T20:59:11.809 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f6067fff700 1 -- 172.21.15.96:0/3030440519 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6074011db0 con 0x562ddb5d1000 2024-04-25T20:59:11.810 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f6067fff700 1 -- 172.21.15.96:0/3030440519 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f607401eb10 con 0x562ddb5d1000 2024-04-25T20:59:11.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 -- 172.21.15.96:0/3030440519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 msgr2=0x562ddb5e7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:11.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5e7890 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f607400ff70 tx=0x7f607400ffa0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 -- 172.21.15.96:0/3030440519 shutdown_connections 2024-04-25T20:59:11.812 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5dbf00 0x562ddb5f3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d06e0 0x562ddb5d9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3030440519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5d1000 0x562ddb5e7890 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 -- 172.21.15.96:0/3030440519 >> 172.21.15.96:0/3030440519 conn(0x562ddb4fda20 msgr2=0x562ddb5cbcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 -- 172.21.15.96:0/3030440519 shutdown_connections 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 -- 172.21.15.96:0/3030440519 wait complete. 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.808+0000 7f608f71eb80 1 -- start start 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 Processor -- start 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d06e0 0x562ddb64b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d1000 0x562ddb64b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5dff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60801ab630 con 0x562ddb5dbf00 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f60801acaa0 con 0x562ddb5d06e0 2024-04-25T20:59:11.813 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f60801abd00 con 0x562ddb5d1000 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f608f71eb80 10 client.0 _ll_register_callbacks cb 0x562ddb4bcab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5dff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5dff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45552/0 (socket says 172.21.15.96:45552) 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 -- 172.21.15.96:0/3092849265 learned_addr learned my addr 172.21.15.96:0/3092849265 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607e59c700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d06e0 0x562ddb64b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607dd9b700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d1000 0x562ddb64b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 -- 172.21.15.96:0/3092849265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d1000 msgr2=0x562ddb64b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d1000 0x562ddb64b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 -- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d06e0 msgr2=0x562ddb64b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d06e0 0x562ddb64b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 -- 172.21.15.96:0/3092849265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ddb5e0660 con 0x562ddb5dbf00 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607dd9b700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d1000 0x562ddb64b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:11.814 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f607ed9d700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5dff90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6078004c40 tx=0x7f6078013fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8bae02406ba0115 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.809+0000 7f6065ffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f6065ffb700 1 -- 172.21.15.96:0/3092849265 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6078004e70 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f6065ffb700 1 -- 172.21.15.96:0/3092849265 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f60780058e0 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6074011db0 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f6065ffb700 1 -- 172.21.15.96:0/3092849265 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f60780076f0 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 10 client.15531 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562ddb652860 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f6065ffb700 1 -- 172.21.15.96:0/3092849265 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6074011db0 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f6065ffb700 1 -- 172.21.15.96:0/3092849265 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7f6078030070 con 0x562ddb5dbf00 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 10 client.15531 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 10 client.15531 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 20 client.15531 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb654fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:11.815 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 4 client.15531 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}] 2024-04-25T20:59:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.2, "await_for": 42.0}) -- 0x562ddb52b6b0 con 0x562ddb5eb820 2024-04-25T20:59:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.810+0000 7f607e59c700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb654fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.811+0000 7f6065ffb700 10 client.15531 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:11.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:11.811+0000 7f607e59c700 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb654fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e138c507298ede server_cookie=dd78ecba661b5113 in_seq=0 out_seq=0 2024-04-25T20:59:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:12 smithi096 ceph-mon[29288]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-25T20:59:12.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:12 smithi096 ceph-mon[29288]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T20:59:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:12 smithi097 ceph-mon[32667]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-25T20:59:12.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:12 smithi097 ceph-mon[32667]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T20:59:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:12 smithi139 ceph-mon[33027]: pgmap v49: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 28 op/s 2024-04-25T20:59:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:12 smithi139 ceph-mon[33027]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-04-25T20:59:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 -- 172.21.15.96:0/3634765728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768101b60 msgr2=0x7fe76810dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 --2- 172.21.15.96:0/3634765728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768101b60 0x7fe76810dc70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fe758009900 tx=0x7fe75802a400 comp rx=0 tx=0).stop 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 -- 172.21.15.96:0/3634765728 shutdown_connections 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 --2- 172.21.15.96:0/3634765728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768101b60 0x7fe76810dc70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 --2- 172.21.15.96:0/3634765728 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe7681011a0 0x7fe768101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 --2- 172.21.15.96:0/3634765728 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe768106fb0 0x7fe768107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 -- 172.21.15.96:0/3634765728 >> 172.21.15.96:0/3634765728 conn(0x7fe768076c50 msgr2=0x7fe768077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 -- 172.21.15.96:0/3634765728 shutdown_connections 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.274+0000 7fe76edad640 1 -- 172.21.15.96:0/3634765728 wait complete. 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 Processor -- start 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 -- start start 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7681011a0 0x7fe768197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe768101b60 0x7fe768197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 0x7fe76819bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe768111160 con 0x7fe768106fb0 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe768110fe0 con 0x7fe768101b60 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76edad640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681112e0 con 0x7fe7681011a0 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 0x7fe76819bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 0x7fe76819bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45566/0 (socket says 172.21.15.96:45566) 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 -- 172.21.15.96:0/3644807963 learned_addr learned my addr 172.21.15.96:0/3644807963 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76cb49640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7681011a0 0x7fe768197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe767fff640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe768101b60 0x7fe768197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 -- 172.21.15.96:0/3644807963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7681011a0 msgr2=0x7fe768197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7681011a0 0x7fe768197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 -- 172.21.15.96:0/3644807963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe768101b60 msgr2=0x7fe768197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe768101b60 0x7fe768197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe76d34a640 1 -- 172.21.15.96:0/3644807963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76819c5f0 con 0x7fe768106fb0 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.275+0000 7fe767fff640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe768101b60 0x7fe768197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.276+0000 7fe76d34a640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 0x7fe76819bec0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe758009a20 tx=0x7fe758004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.276+0000 7fe765ffb640 1 -- 172.21.15.96:0/3644807963 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75803f030 con 0x7fe768106fb0 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.276+0000 7fe765ffb640 1 -- 172.21.15.96:0/3644807963 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe758003aa0 con 0x7fe768106fb0 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.276+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76819c880 con 0x7fe768106fb0 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.276+0000 7fe765ffb640 1 -- 172.21.15.96:0/3644807963 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe758035070 con 0x7fe768106fb0 2024-04-25T20:59:13.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.276+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a4170 con 0x7fe768106fb0 2024-04-25T20:59:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe765ffb640 1 -- 172.21.15.96:0/3644807963 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe758042030 con 0x7fe768106fb0 2024-04-25T20:59:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 Processor -- start 2024-04-25T20:59:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 -- start start 2024-04-25T20:59:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 0x5612f3e2c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e2ca10 0x5612f3e2ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612f3d84970 con 0x5612f3e1fda0 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612f3d214a0 con 0x5612f3e1f480 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe771ba6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5612f3d80930 con 0x5612f3e2ca10 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe765ffb640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe7540a3dc0 0x7fe7540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe76cb49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 0x5612f3e2c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe76cb49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 0x5612f3e2c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45570/0 (socket says 172.21.15.96:45570) 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe76cb49640 1 -- 172.21.15.96:0/1771112065 learned_addr learned my addr 172.21.15.96:0/1771112065 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe76d34a640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe7540a3dc0 0x7fe7540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe767fff640 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.279+0000 7fe765ffb640 1 -- 172.21.15.96:0/3644807963 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe7580f0d60 con 0x7fe768106fb0 2024-04-25T20:59:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe767fff640 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e2ca10 0x5612f3e2ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76cb49640 1 -- 172.21.15.96:0/1771112065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e2ca10 msgr2=0x5612f3e2ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76cb49640 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e2ca10 0x5612f3e2ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76cb49640 1 -- 172.21.15.96:0/1771112065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 msgr2=0x5612f3e1f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76cb49640 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e1f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76cb49640 1 -- 172.21.15.96:0/1771112065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f3a97210 con 0x5612f3e1fda0 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe767fff640 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e2ca10 0x5612f3e2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76d34a640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe7540a3dc0 0x7fe7540a6280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7580eeb20 tx=0x7fe75808e7b0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe76cb49640 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 0x5612f3e2c4d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe76000a4b0 tx=0x7fe7600096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3e75cb9e57f045 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe75effd640 1 -- 172.21.15.96:0/1771112065 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760018020 con 0x5612f3e1fda0 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe75effd640 1 -- 172.21.15.96:0/1771112065 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe760014cc0 con 0x5612f3e1fda0 2024-04-25T20:59:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.280+0000 7fe75effd640 1 -- 172.21.15.96:0/1771112065 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760013c70 con 0x5612f3e1fda0 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 -- 172.21.15.96:0/1771112065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 msgr2=0x5612f3e2c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 0x5612f3e2c4d0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe76000a4b0 tx=0x7fe7600096d0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 -- 172.21.15.96:0/1771112065 shutdown_connections 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e2ca10 0x5612f3e2ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e1fda0 0x5612f3e2c4d0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 --2- 172.21.15.96:0/1771112065 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e1f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 -- 172.21.15.96:0/1771112065 >> 172.21.15.96:0/1771112065 conn(0x5612f3e1bb30 msgr2=0x5612f3e1bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 -- 172.21.15.96:0/1771112065 shutdown_connections 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 -- 172.21.15.96:0/1771112065 wait complete. 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 -- start start 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.281+0000 7fe771ba6740 1 Processor -- start 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e2c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.290 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e295c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e29b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612f3e2f640 con 0x5612f3e2ca10 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5612f3d1f0b0 con 0x5612f3e1f480 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5612f3e2f7c0 con 0x5612f3e1fda0 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 10 client.0 _ll_register_callbacks cb 0x5612f3d0cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e29b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e29b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45576/0 (socket says 172.21.15.96:45576) 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 -- 172.21.15.96:0/2542096236 learned_addr learned my addr 172.21.15.96:0/2542096236 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e2c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe76cb49640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e295c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 -- 172.21.15.96:0/2542096236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 msgr2=0x5612f3e295c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e295c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 -- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 msgr2=0x5612f3e2c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e2c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 -- 172.21.15.96:0/2542096236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f3b63620 con 0x5612f3e2ca10 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe76cb49640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe767fff640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e29b00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe750019de0 tx=0x7fe7500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=293b4d8543d02509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe747fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe747fff640 1 -- 172.21.15.96:0/2542096236 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750012d70 con 0x5612f3e2ca10 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe747fff640 1 -- 172.21.15.96:0/2542096236 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe750016400 con 0x5612f3e2ca10 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe760014cc0 con 0x5612f3e2ca10 2024-04-25T20:59:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe747fff640 1 -- 172.21.15.96:0/2542096236 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe750016600 con 0x5612f3e2ca10 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 10 client.15543 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.282+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612f3e3b5b0 con 0x5612f3e2ca10 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe747fff640 1 -- 172.21.15.96:0/2542096236 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe760014cc0 con 0x5612f3e2ca10 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe747fff640 1 -- 172.21.15.96:0/2542096236 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe750015040 con 0x5612f3e2ca10 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe771ba6740 10 client.15543 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe771ba6740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe771ba6740 20 client.15543 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3e3ca70 0x5612f3e3ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe771ba6740 4 client.15543 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612f3d92390 con 0x5612f3e3ca70 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe76cb49640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3e3ca70 0x5612f3e3ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe747fff640 10 client.15543 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.283+0000 7fe76cb49640 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3e3ca70 0x5612f3e3ee60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced8172ade551817 server_cookie=3adfc4c2865f2769 in_seq=0 out_seq=0 2024-04-25T20:59:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe747fff640 1 -- 172.21.15.96:0/2542096236 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612f3d92390 con 0x5612f3e3ca70 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe747fff640 10 client.15543 handle_command_reply: tid=0 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 client.15543 shutdown 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 msgr2=0x5612f3e29b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e29b00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe750019de0 tx=0x7fe7500046c0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 shutdown_connections 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3e3ca70 0x5612f3e3ee60 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e29b00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e295c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe747fff640 0 client.15543 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2542096236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e2c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 >> 172.21.15.96:0/2542096236 conn(0x5612f3e1bb30 msgr2=0x5612f3e21900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 shutdown_connections 2024-04-25T20:59:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.284+0000 7fe771ba6740 1 -- 172.21.15.96:0/2542096236 wait complete. 2024-04-25T20:59:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.285+0000 7fe771ba6740 20 client.15543 trim_cache size 0 max 16384 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 Processor -- start 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 -- start start 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e38540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e24b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e2be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ae0f0 con 0x5612f3e2ca10 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ab9c0 con 0x5612f3e1f480 2024-04-25T20:59:13.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe771ba6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681a9830 con 0x5612f3e1fda0 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e2be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e2be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45582/0 (socket says 172.21.15.96:45582) 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 -- 172.21.15.96:0/2928901568 learned_addr learned my addr 172.21.15.96:0/2928901568 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 -- 172.21.15.96:0/2928901568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 msgr2=0x5612f3e24b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe76cb49640 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e24b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e24b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 -- 172.21.15.96:0/2928901568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 msgr2=0x5612f3e38540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e38540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.299+0000 7fe767fff640 1 -- 172.21.15.96:0/2928901568 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612f3e17500 con 0x5612f3e2ca10 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe767fff640 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e2be00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe750001eb0 tx=0x7fe7500198d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776c40576a392289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe75e7fc640 1 -- 172.21.15.96:0/2928901568 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75000ae40 con 0x5612f3e2ca10 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe75e7fc640 1 -- 172.21.15.96:0/2928901568 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe750016400 con 0x5612f3e2ca10 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe75e7fc640 1 -- 172.21.15.96:0/2928901568 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe750016c50 con 0x5612f3e2ca10 2024-04-25T20:59:13.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 -- 172.21.15.96:0/2928901568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 msgr2=0x5612f3e2be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e2be00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fe750001eb0 tx=0x7fe7500198d0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 -- 172.21.15.96:0/2928901568 shutdown_connections 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e2be00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e24b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 --2- 172.21.15.96:0/2928901568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e38540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 -- 172.21.15.96:0/2928901568 >> 172.21.15.96:0/2928901568 conn(0x5612f3e1bb30 msgr2=0x5612f3e24470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 -- 172.21.15.96:0/2928901568 shutdown_connections 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.300+0000 7fe771ba6740 1 -- 172.21.15.96:0/2928901568 wait complete. 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 -- start start 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 Processor -- start 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e34c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e25eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e263f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681a9e80 con 0x5612f3e2ca10 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ab2f0 con 0x5612f3e1f480 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681aa550 con 0x5612f3e1fda0 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe771ba6740 10 client.0 _ll_register_callbacks cb 0x7fe7500199f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e263f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e263f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45592/0 (socket says 172.21.15.96:45592) 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 -- 172.21.15.96:0/4077887102 learned_addr learned my addr 172.21.15.96:0/4077887102 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 -- 172.21.15.96:0/4077887102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 msgr2=0x5612f3e25eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe76cb49640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e25eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e25eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 -- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 msgr2=0x5612f3e34c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e34c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe767fff640 1 -- 172.21.15.96:0/4077887102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612f3e26960 con 0x5612f3e2ca10 2024-04-25T20:59:13.307 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.301+0000 7fe76cb49640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e25eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe767fff640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e263f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a690 tx=0x7fe75000af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e93f6f7f62683abf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe747fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe747fff640 1 -- 172.21.15.96:0/4077887102 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7500233e0 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe747fff640 1 -- 172.21.15.96:0/4077887102 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fe7500238f0 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe750016400 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe747fff640 1 -- 172.21.15.96:0/4077887102 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fe750023af0 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5612f3ea2d50 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe747fff640 1 -- 172.21.15.96:0/4077887102 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe750016400 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe747fff640 1 -- 172.21.15.96:0/4077887102 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8338+0+0 (secure 0 0 0) 0x7fe75002c070 con 0x5612f3e2ca10 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T20:59:13.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 20 client.15549 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3ea3300 0x5612f3ea56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 4 client.15549 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T20:59:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.302+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5612f3e29e90 con 0x5612f3ea3300 2024-04-25T20:59:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.303+0000 7fe76cb49640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3ea3300 0x5612f3ea56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.303+0000 7fe747fff640 10 client.15549 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:13.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T20:59:13.303+0000 7fe76cb49640 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3ea3300 0x5612f3ea56f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45ac9501745ede8 server_cookie=a41df2b143db6fb2 in_seq=0 out_seq=0 2024-04-25T20:59:14.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:14 smithi096 ceph-mon[29288]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 26 op/s 2024-04-25T20:59:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:14 smithi097 ceph-mon[32667]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 26 op/s 2024-04-25T20:59:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:14 smithi139 ceph-mon[33027]: pgmap v50: 97 pgs: 97 active+clean; 15 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 26 op/s 2024-04-25T20:59:15.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:15 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:59:15.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:15 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:59:15.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:15 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T20:59:16.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:16 smithi096 ceph-mon[29288]: pgmap v51: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-04-25T20:59:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:16 smithi097 ceph-mon[32667]: pgmap v51: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-04-25T20:59:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:16 smithi139 ceph-mon[33027]: pgmap v51: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 146 op/s 2024-04-25T20:59:18.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:18 smithi096 ceph-mon[29288]: pgmap v52: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-04-25T20:59:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:18 smithi097 ceph-mon[32667]: pgmap v52: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-04-25T20:59:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:18 smithi139 ceph-mon[33027]: pgmap v52: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-04-25T20:59:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:20 smithi096 ceph-mon[29288]: pgmap v53: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-04-25T20:59:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:20 smithi097 ceph-mon[32667]: pgmap v53: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-04-25T20:59:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:20 smithi139 ceph-mon[33027]: pgmap v53: 97 pgs: 97 active+clean; 17 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.4 MiB/s wr, 128 op/s 2024-04-25T20:59:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:22 smithi096 ceph-mon[29288]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T20:59:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:22 smithi097 ceph-mon[32667]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T20:59:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:22 smithi139 ceph-mon[33027]: pgmap v54: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 131 op/s 2024-04-25T20:59:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:24 smithi096 ceph-mon[29288]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 204 KiB/s wr, 122 op/s 2024-04-25T20:59:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:24 smithi097 ceph-mon[32667]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 204 KiB/s wr, 122 op/s 2024-04-25T20:59:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:24 smithi139 ceph-mon[33027]: pgmap v55: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 204 KiB/s wr, 122 op/s 2024-04-25T20:59:26.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:26 smithi096 ceph-mon[29288]: pgmap v56: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 207 KiB/s wr, 126 op/s 2024-04-25T20:59:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:26 smithi097 ceph-mon[32667]: pgmap v56: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 207 KiB/s wr, 126 op/s 2024-04-25T20:59:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:26 smithi139 ceph-mon[33027]: pgmap v56: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 207 KiB/s wr, 126 op/s 2024-04-25T20:59:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:28 smithi096 ceph-mon[29288]: pgmap v57: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-04-25T20:59:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:28 smithi097 ceph-mon[32667]: pgmap v57: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-04-25T20:59:28.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:28 smithi139 ceph-mon[33027]: pgmap v57: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-04-25T20:59:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:30 smithi096 ceph-mon[29288]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-04-25T20:59:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:30 smithi097 ceph-mon[32667]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-04-25T20:59:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:30 smithi139 ceph-mon[33027]: pgmap v58: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 15 KiB/s wr, 6 op/s 2024-04-25T20:59:32.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:32 smithi096 ceph-mon[29288]: pgmap v59: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-04-25T20:59:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:32 smithi097 ceph-mon[32667]: pgmap v59: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-04-25T20:59:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:32 smithi139 ceph-mon[33027]: pgmap v59: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 15 KiB/s wr, 9 op/s 2024-04-25T20:59:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:34 smithi096 ceph-mon[29288]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-25T20:59:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:34 smithi097 ceph-mon[32667]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-25T20:59:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:34 smithi139 ceph-mon[33027]: pgmap v60: 97 pgs: 97 active+clean; 17 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 3.2 KiB/s wr, 6 op/s 2024-04-25T20:59:36.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:36 smithi096 ceph-mon[29288]: pgmap v61: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 299 KiB/s wr, 47 op/s 2024-04-25T20:59:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:36 smithi097 ceph-mon[32667]: pgmap v61: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 299 KiB/s wr, 47 op/s 2024-04-25T20:59:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:36 smithi139 ceph-mon[33027]: pgmap v61: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 299 KiB/s wr, 47 op/s 2024-04-25T20:59:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:38 smithi096 ceph-mon[29288]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 295 KiB/s wr, 43 op/s 2024-04-25T20:59:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:38 smithi097 ceph-mon[32667]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 295 KiB/s wr, 43 op/s 2024-04-25T20:59:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:38 smithi139 ceph-mon[33027]: pgmap v62: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 295 KiB/s wr, 43 op/s 2024-04-25T20:59:40.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:40 smithi096 ceph-mon[29288]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 295 KiB/s wr, 43 op/s 2024-04-25T20:59:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:40 smithi097 ceph-mon[32667]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 295 KiB/s wr, 43 op/s 2024-04-25T20:59:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:40 smithi139 ceph-mon[33027]: pgmap v63: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 295 KiB/s wr, 43 op/s 2024-04-25T20:59:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:42 smithi096 ceph-mon[29288]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 297 KiB/s wr, 46 op/s 2024-04-25T20:59:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:42 smithi097 ceph-mon[32667]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 297 KiB/s wr, 46 op/s 2024-04-25T20:59:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:42 smithi139 ceph-mon[33027]: pgmap v64: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 297 KiB/s wr, 46 op/s 2024-04-25T20:59:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:44 smithi096 ceph-mon[29288]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 297 KiB/s wr, 44 op/s 2024-04-25T20:59:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:44 smithi097 ceph-mon[32667]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 297 KiB/s wr, 44 op/s 2024-04-25T20:59:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:44 smithi139 ceph-mon[33027]: pgmap v65: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 297 KiB/s wr, 44 op/s 2024-04-25T20:59:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:46 smithi096 ceph-mon[29288]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 299 KiB/s wr, 47 op/s 2024-04-25T20:59:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:46.500 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:59:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:46 smithi097 ceph-mon[32667]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 299 KiB/s wr, 47 op/s 2024-04-25T20:59:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:59:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:46 smithi139 ceph-mon[33027]: pgmap v66: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 299 KiB/s wr, 47 op/s 2024-04-25T20:59:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T20:59:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T20:59:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T20:59:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T20:59:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:48 smithi097 ceph-mon[32667]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-25T20:59:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:48 smithi139 ceph-mon[33027]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-25T20:59:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:48 smithi096 ceph-mon[29288]: pgmap v67: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-25T20:59:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:50 smithi097 ceph-mon[32667]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-25T20:59:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:50 smithi139 ceph-mon[33027]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-25T20:59:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:50 smithi096 ceph-mon[29288]: pgmap v68: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.4 KiB/s wr, 5 op/s 2024-04-25T20:59:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:52 smithi097 ceph-mon[32667]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-25T20:59:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:52 smithi139 ceph-mon[33027]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-25T20:59:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:52 smithi096 ceph-mon[29288]: pgmap v69: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 27, 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "leader": 24443, 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "set_version": 5, 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "sets": { 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "2c77974d": { 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "version": 5, 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age_ref": 0.0, 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "QUIESCING", 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.0 2024-04-25T20:59:53.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "timeout": 60.0, 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "expiration": 86.2, 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "members": { 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "file:/": { 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "excluded": false, 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "QUIESCING", 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 42.0 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T20:59:53.815 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f6065ffb700 1 -- 172.21.15.96:0/3092849265 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: -115 ) ==== 8+0+592 (crc 0 0 0) 0x562ddb52b6b0 con 0x562ddb5eb820 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f6065ffb700 10 client.15531 handle_command_reply: tid=0 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 client.15531 shutdown 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 msgr2=0x562ddb5dff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5dff90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6078004c40 tx=0x7f6078013fb0 comp rx=0 tx=0).stop 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 shutdown_connections 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ddb5dbf00 0x562ddb5dff90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562ddb5d06e0 0x562ddb64b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562ddb5eb820 0x562ddb654fe0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 --2- 172.21.15.96:0/3092849265 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562ddb5d1000 0x562ddb64b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f6065ffb700 0 client.15531 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.811+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 >> 172.21.15.96:0/3092849265 conn(0x562ddb4fda20 msgr2=0x562ddb5e6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:53.816 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.812+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 shutdown_connections 2024-04-25T20:59:53.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.812+0000 7f608f71eb80 1 -- 172.21.15.96:0/3092849265 wait complete. 2024-04-25T20:59:53.817 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:53.812+0000 7f608f71eb80 20 client.15531 trim_cache size 0 max 16384 2024-04-25T20:59:53.817 INFO:teuthology.orchestra.run.smithi096.stderr:Error EINPROGRESS: 2024-04-25T20:59:54.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:54 smithi096 ceph-mon[29288]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T20:59:54.572 DEBUG:teuthology.orchestra.run:got remote process result: 115 2024-04-25T20:59:54.573 WARNING:tasks.quiescer.fs.[cephfs]:Set '2c77974d' hasn't quiesced after 47.6 seconds (timeout: 60). Dumping ops with locks from all ranks. 2024-04-25T20:59:54.573 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T20:59:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:54 smithi097 ceph-mon[32667]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T20:59:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:54 smithi139 ceph-mon[33027]: pgmap v70: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-04-25T20:59:54.818 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 -- 172.21.15.96:0/1401613896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc101340 msgr2=0x7f98cc101720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1401613896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc101340 0x7f98cc101720 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f98c8009990 tx=0x7f98c8009ca0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 -- 172.21.15.96:0/1401613896 shutdown_connections 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1401613896 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc106420 0x7f98cc10dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1401613896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc101340 0x7f98cc101720 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1401613896 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101cf0 0x7f98cc105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 -- 172.21.15.96:0/1401613896 >> 172.21.15.96:0/1401613896 conn(0x7f98cc077f40 msgr2=0x7f98cc0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:56.416 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 -- 172.21.15.96:0/1401613896 shutdown_connections 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.413+0000 7f98d35b2700 1 -- 172.21.15.96:0/1401613896 wait complete. 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 Processor -- start 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 -- start start 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101340 0x7f98cc1991c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc101cf0 0x7f98cc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 0x7f98cc19da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112cb0 con 0x7f98cc106420 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112b30 con 0x7f98cc101cf0 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d35b2700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc112e30 con 0x7f98cc101340 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 0x7f98cc19da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 0x7f98cc19da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47366/0 (socket says 172.21.15.96:47366) 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 -- 172.21.15.96:0/1507725040 learned_addr learned my addr 172.21.15.96:0/1507725040 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:56.417 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d0b4d700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc101cf0 0x7f98cc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d134e700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101340 0x7f98cc1991c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 -- 172.21.15.96:0/1507725040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101340 msgr2=0x7f98cc1991c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101340 0x7f98cc1991c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 -- 172.21.15.96:0/1507725040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc101cf0 msgr2=0x7f98cc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc101cf0 0x7f98cc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 -- 172.21.15.96:0/1507725040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc19e1d0 con 0x7f98cc106420 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d134e700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101340 0x7f98cc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.414+0000 7f98d1b4f700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 0x7f98cc19da90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f98c4007f10 tx=0x7f98c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.415+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4009e20 con 0x7f98cc106420 2024-04-25T20:59:56.418 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.415+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc19e4c0 con 0x7f98cc106420 2024-04-25T20:59:56.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.415+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c400dbd0 con 0x7f98cc106420 2024-04-25T20:59:56.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.415+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4005990 con 0x7f98cc106420 2024-04-25T20:59:56.419 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.415+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a5d40 con 0x7f98cc106420 2024-04-25T20:59:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.416+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc069730 con 0x7f98cc106420 2024-04-25T20:59:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.417+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f98c401a020 con 0x7f98cc106420 2024-04-25T20:59:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.417+0000 7f98c27fc700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f98b80a3db0 0x7f98b80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.418+0000 7f98d134e700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f98b80a3db0 0x7f98b80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.418+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f98c40cffd0 con 0x7f98cc106420 2024-04-25T20:59:56.420 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.418+0000 7f98d134e700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f98b80a3db0 0x7f98b80a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f98c8006010 tx=0x7f98c800b540 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:56.423 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.422+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98c4010070 con 0x7f98cc106420 2024-04-25T20:59:56.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:56 smithi096 ceph-mon[29288]: pgmap v71: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-25T20:59:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:56 smithi097 ceph-mon[32667]: pgmap v71: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-25T20:59:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:56 smithi139 ceph-mon[33027]: pgmap v71: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.8 KiB/s wr, 8 op/s 2024-04-25T20:59:56.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.667+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98cc02df80 con 0x7f98cc106420 2024-04-25T20:59:56.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.668+0000 7f98c27fc700 1 -- 172.21.15.96:0/1507725040 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11862 (secure 0 0 0) 0x7f98c4022250 con 0x7f98cc106420 2024-04-25T20:59:56.669 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T20:59:56.669 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:59:13.489761+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f98b80a3db0 msgr2=0x7f98b80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f98b80a3db0 0x7f98b80a6270 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f98c8006010 tx=0x7f98c800b540 comp rx=0 tx=0).stop 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 msgr2=0x7f98cc19da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 0x7f98cc19da90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f98c4007f10 tx=0x7f98c400d3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 shutdown_connections 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f98b80a3db0 0x7f98b80a6270 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc106420 0x7f98cc19da90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f98cc101340 0x7f98cc1991c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 --2- 172.21.15.96:0/1507725040 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f98cc101cf0 0x7f98cc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:56.671 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 >> 172.21.15.96:0/1507725040 conn(0x7f98cc077f40 msgr2=0x7f98cc0fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 shutdown_connections 2024-04-25T20:59:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:56.670+0000 7f98d35b2700 1 -- 172.21.15.96:0/1507725040 wait complete. 2024-04-25T20:59:56.672 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-04-25T20:59:57.424 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 2 (h) to a remote file /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.h.json 2024-04-25T20:59:57.424 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.1:2 ops --flags=locks --path=/var/run/ceph/ops-2c77974d-1-mds.h.json 2024-04-25T20:59:57.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:57 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1507725040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:57.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:57 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1507725040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:57.669 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T20:59:57.706 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:57 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1507725040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T20:59:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 20:59:58 smithi096 ceph-mon[29288]: pgmap v72: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T20:59:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 20:59:58 smithi097 ceph-mon[32667]: pgmap v72: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T20:59:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 20:59:58 smithi139 ceph-mon[33027]: pgmap v72: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 -- 172.21.15.96:0/3858762028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 msgr2=0x7f6dbc10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 --2- 172.21.15.96:0/3858762028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc10e7f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b0a0 tx=0x7f6db400b3b0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 -- 172.21.15.96:0/3858762028 shutdown_connections 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 --2- 172.21.15.96:0/3858762028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc10e7f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 --2- 172.21.15.96:0/3858762028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dbc107990 0x7f6dbc107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 --2- 172.21.15.96:0/3858762028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 0x7f6dbc101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 -- 172.21.15.96:0/3858762028 >> 172.21.15.96:0/3858762028 conn(0x7f6dbc0788f0 msgr2=0x7f6dbc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 -- 172.21.15.96:0/3858762028 shutdown_connections 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.351+0000 7f6dc158e700 1 -- 172.21.15.96:0/3858762028 wait complete. 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 Processor -- start 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 -- start start 2024-04-25T20:59:59.364 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 0x7f6dbc199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc1996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dbc107990 0x7f6dbc19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc111b50 con 0x7f6dbc102300 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1119d0 con 0x7f6dbc107990 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dc158e700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc111cd0 con 0x7f6dbc101940 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc1996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc1996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35810/0 (socket says 172.21.15.96:35810) 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 -- 172.21.15.96:0/2492943338 learned_addr learned my addr 172.21.15.96:0/2492943338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dbc107990 0x7f6dbc19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 0x7f6dbc199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 -- 172.21.15.96:0/2492943338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 msgr2=0x7f6dbc199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 0x7f6dbc199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 -- 172.21.15.96:0/2492943338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dbc107990 msgr2=0x7f6dbc19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dbc107990 0x7f6dbc19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.365 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 -- 172.21.15.96:0/2492943338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dbc19e120 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 0x7f6dbc199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.352+0000 7f6db3fff700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc1996a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b530 tx=0x7f6da400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.353+0000 7f6db8ff9700 1 -- 172.21.15.96:0/2492943338 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4015020 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.353+0000 7f6db8ff9700 1 -- 172.21.15.96:0/2492943338 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6da4011ce0 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.353+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dbc19e410 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.353+0000 7f6db8ff9700 1 -- 172.21.15.96:0/2492943338 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6da4010e40 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.353+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dbc1a5c90 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.355+0000 7f6db8ff9700 1 -- 172.21.15.96:0/2492943338 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6da4018020 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.355+0000 7f6db8ff9700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6da80a3db0 0x7f6da80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.355+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6da80a3db0 0x7f6da80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.355+0000 7f6db8ff9700 1 -- 172.21.15.96:0/2492943338 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6da40cfef0 con 0x7f6dbc102300 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6da80a3db0 0x7f6da80a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005950 tx=0x7f6dac0058e0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:59.366 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 Processor -- start 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 -- start start 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabba9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbbb9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dcabafce10 con 0x55dcabbb95d0 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dcabaa3a40 con 0x55dcabba99d0 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.356+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dcabaf8dd0 con 0x55dcabbaa2f0 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbbb9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbbb9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35818/0 (socket says 172.21.15.96:35818) 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/2299127211 learned_addr learned my addr 172.21.15.96:0/2299127211 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6db3fff700 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabba9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/2299127211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 msgr2=0x55dcabbb9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.367 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/2299127211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 msgr2=0x55dcabba9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabba9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/2299127211 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcaba4a9b0 con 0x55dcabbb95d0 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbbb9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009230 tx=0x7f6db4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4338779953baf5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6db17fa700 1 -- 172.21.15.96:0/2299127211 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4020070 con 0x55dcabbb95d0 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6db17fa700 1 -- 172.21.15.96:0/2299127211 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4003c00 con 0x55dcabbb95d0 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.357+0000 7f6db17fa700 1 -- 172.21.15.96:0/2299127211 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4005040 con 0x55dcabbb95d0 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/2299127211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 msgr2=0x55dcabbbb9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbbb9c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f6db4009230 tx=0x7f6db4009260 comp rx=0 tx=0).stop 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/2299127211 shutdown_connections 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbbb9c0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabba9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/2299127211 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/2299127211 >> 172.21.15.96:0/2299127211 conn(0x55dcabad7770 msgr2=0x55dcabba5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:59.368 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/2299127211 shutdown_connections 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/2299127211 wait complete. 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.358+0000 7f6dcbe17b80 1 -- start start 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 Processor -- start 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabbb8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb6050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbb6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dcabbbbf60 con 0x55dcabbb95d0 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55dcaba9bcb0 con 0x55dcabba99d0 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55dcabbbc0e0 con 0x55dcabbaa2f0 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dcbe17b80 10 client.0 _ll_register_callbacks cb 0x55dcaba95a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbb6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbb6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35826/0 (socket says 172.21.15.96:35826) 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/980430733 learned_addr learned my addr 172.21.15.96:0/980430733 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:59.369 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabbb8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6db3fff700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/980430733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 msgr2=0x55dcabbb6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb6050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 msgr2=0x55dcabbb8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabbb8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/980430733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcabab4350 con 0x55dcabbb95d0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabbb8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6db3fff700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbb6590 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b3f0 tx=0x7f6db4003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7905b35ae6a6d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.359+0000 7f6da27fc700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6da27fc700 1 -- 172.21.15.96:0/980430733 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4020040 con 0x55dcabbb95d0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6da27fc700 1 -- 172.21.15.96:0/980430733 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6db4007930 con 0x55dcabbb95d0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6db4003c00 con 0x55dcabbb95d0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6da27fc700 1 -- 172.21.15.96:0/980430733 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6db40050c0 con 0x55dcabbb95d0 2024-04-25T20:59:59.370 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 10 client.15567 fetch_fsmap learned FSMap version 27 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcabbc67d0 con 0x55dcabbb95d0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6da27fc700 1 -- 172.21.15.96:0/980430733 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6db4003c00 con 0x55dcabbb95d0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6da27fc700 1 -- 172.21.15.96:0/980430733 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6db4036050 con 0x55dcabbb95d0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 10 client.15567 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 20 client.15567 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabbcb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 4 client.15567 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcabb0a600 con 0x55dcabbc8ca0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.360+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabbcb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.361+0000 7f6da27fc700 10 client.15567 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.361+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabbcb090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=18f2a53c60b2143b server_cookie=7cb35e58d52298e2 in_seq=0 out_seq=0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6da27fc700 1 -- 172.21.15.96:0/980430733 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dcabb0a600 con 0x55dcabbc8ca0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6da27fc700 10 client.15567 handle_command_reply: tid=0 2024-04-25T20:59:59.371 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 client.15567 shutdown 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 msgr2=0x55dcabbb6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbb6590 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f6db400b3f0 tx=0x7f6db4003a00 comp rx=0 tx=0).stop 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 shutdown_connections 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabbcb090 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6da27fc700 0 client.15567 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabbb6590 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabba99d0 0x55dcabbb8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/980430733 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 >> 172.21.15.96:0/980430733 conn(0x55dcabad7770 msgr2=0x55dcabba5ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 shutdown_connections 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/980430733 wait complete. 2024-04-25T20:59:59.372 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.362+0000 7f6dcbe17b80 20 client.15567 trim_cache size 0 max 16384 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.381+0000 7f6dcbe17b80 1 Processor -- start 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.381+0000 7f6dcbe17b80 1 -- start start 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 0x55dcabbb1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbb95d0 0x55dcabbb3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1ad780 con 0x55dcabba99d0 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1ab010 con 0x55dcabbb95d0 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1a8ea0 con 0x55dcabbaa2f0 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 0x55dcabbb1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 0x55dcabbb1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35842/0 (socket says 172.21.15.96:35842) 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 -- 172.21.15.96:0/1365062520 learned_addr learned my addr 172.21.15.96:0/1365062520 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6db3fff700 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbb95d0 0x55dcabbb3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 -- 172.21.15.96:0/1365062520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 msgr2=0x55dcabbb3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 -- 172.21.15.96:0/1365062520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbb95d0 msgr2=0x55dcabbb3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbb95d0 0x55dcabbb3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 -- 172.21.15.96:0/1365062520 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcabbb5e20 con 0x55dcabba99d0 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbb95d0 0x55dcabbb3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 0x55dcabbb1740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c010 tx=0x7f6dac015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f2b73cfded39dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6db0ff9700 1 -- 172.21.15.96:0/1365062520 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac02c070 con 0x55dcabba99d0 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6db0ff9700 1 -- 172.21.15.96:0/1365062520 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac0281c0 con 0x55dcabba99d0 2024-04-25T20:59:59.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.382+0000 7f6db0ff9700 1 -- 172.21.15.96:0/1365062520 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0038c0 con 0x55dcabba99d0 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/1365062520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 msgr2=0x55dcabbb1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 0x55dcabbb1740 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f6dac00c010 tx=0x7f6dac015000 comp rx=0 tx=0).stop 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/1365062520 shutdown_connections 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbb95d0 0x55dcabbb3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabba99d0 0x55dcabbb1740 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/1365062520 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabbaa2f0 0x55dcabbb3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/1365062520 >> 172.21.15.96:0/1365062520 conn(0x55dcabad7770 msgr2=0x55dcabbb1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:59.387 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/1365062520 shutdown_connections 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/1365062520 wait complete. 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.383+0000 7f6dcbe17b80 1 -- start start 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 Processor -- start 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabba99d0 0x55dcabbcc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbaa2f0 0x55dcabbcc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabc2de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1a94f0 con 0x55dcabbb95d0 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1aa940 con 0x55dcabbaa2f0 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f6dbc1a9ba0 con 0x55dcabba99d0 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dcbe17b80 10 client.0 _ll_register_callbacks cb 0x55dcaba95a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabc2de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabc2de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35844/0 (socket says 172.21.15.96:35844) 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/390383538 learned_addr learned my addr 172.21.15.96:0/390383538 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabba99d0 0x55dcabbcc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.388 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6db3fff700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbaa2f0 0x55dcabbcc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/390383538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabba99d0 msgr2=0x55dcabbcc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabba99d0 0x55dcabbcc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbaa2f0 msgr2=0x55dcabbcc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbaa2f0 0x55dcabbcc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 -- 172.21.15.96:0/390383538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcabc2e3c0 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6db3fff700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbaa2f0 0x55dcabbcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6dbb7fe700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabc2de80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6db402e8d0 tx=0x7f6db4003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f48d4de67fb9451d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.384+0000 7f6da27fc700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6da27fc700 1 -- 172.21.15.96:0/390383538 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6db4020040 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6da27fc700 1 -- 172.21.15.96:0/390383538 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f6db4009180 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dac0281c0 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6da27fc700 1 -- 172.21.15.96:0/390383538 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6db4007940 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 10 client.15573 fetch_fsmap learned FSMap version 27 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55dcabc2f570 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6da27fc700 1 -- 172.21.15.96:0/390383538 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dac0281c0 con 0x55dcabbb95d0 2024-04-25T20:59:59.389 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6da27fc700 1 -- 172.21.15.96:0/390383538 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f6db4036070 con 0x55dcabbb95d0 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 10 client.15573 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 20 client.15573 populate_metadata read hostname 'smithi096' 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabc31cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 4 client.15573 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-2c77974d-1-mds.h.json"}] 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 --> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-2c77974d-1-mds.h.json"}) -- 0x55dcabb046a0 con 0x55dcabbc8ca0 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.385+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabc31cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.386+0000 7f6da27fc700 10 client.15573 ms_handle_connect on v2:172.21.15.97:6836/16128615 2024-04-25T20:59:59.390 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.386+0000 7f6dbaffd700 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabc31cf0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=37cc7d4545378320 server_cookie=d9126d232d3025ea in_seq=0 out_seq=0 2024-04-25T20:59:59.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6da27fc700 1 -- 172.21.15.96:0/390383538 <== mds.2 v2:172.21.15.97:6836/16128615 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55dcabb046a0 con 0x55dcabbc8ca0 2024-04-25T20:59:59.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6da27fc700 10 client.15573 handle_command_reply: tid=0 2024-04-25T20:59:59.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 client.15573 shutdown 2024-04-25T20:59:59.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 msgr2=0x55dcabc2de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabc2de80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f6db402e8d0 tx=0x7f6db4003d30 comp rx=0 tx=0).stop 2024-04-25T20:59:59.391 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 shutdown_connections 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:6836/16128615,v1:172.21.15.97:6837/16128615] conn(0x55dcabbc8ca0 0x55dcabc31cf0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dcabbb95d0 0x55dcabc2de80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55dcabba99d0 0x55dcabbcc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 --2- 172.21.15.96:0/390383538 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55dcabbaa2f0 0x55dcabbcc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 >> 172.21.15.96:0/390383538 conn(0x55dcabad7770 msgr2=0x55dcabbb4980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6da27fc700 0 client.15573 ms_handle_reset on v2:172.21.15.97:6836/16128615 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 shutdown_connections 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 1 -- 172.21.15.96:0/390383538 wait complete. 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.387+0000 7f6dcbe17b80 20 client.15573 trim_cache size 0 max 16384 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/var/run/ceph/ops-2c77974d-1-mds.h.json" 2024-04-25T20:59:59.392 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6da80a3db0 msgr2=0x7f6da80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6da80a3db0 0x7f6da80a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6dac005950 tx=0x7f6dac0058e0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 msgr2=0x7f6dbc1996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc1996a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6da400b530 tx=0x7f6da400b840 comp rx=0 tx=0).stop 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 shutdown_connections 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f6da80a3db0 0x7f6da80a6270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.393 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6dbc107990 0x7f6dbc19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f6dbc101940 0x7f6dbc199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 --2- 172.21.15.96:0/2492943338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6dbc102300 0x7f6dbc1996a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T20:59:59.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 >> 172.21.15.96:0/2492943338 conn(0x7f6dbc0788f0 msgr2=0x7f6dbc06aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T20:59:59.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 shutdown_connections 2024-04-25T20:59:59.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T20:59:59.388+0000 7f6dc158e700 1 -- 172.21.15.96:0/2492943338 wait complete. 2024-04-25T21:00:00.142 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 1 (i) to a remote file /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.i.json 2024-04-25T21:00:00.143 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.1:1 ops --flags=locks --path=/var/run/ceph/ops-2c77974d-1-mds.i.json 2024-04-25T21:00:00.387 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:00 smithi096 ceph-mon[29288]: pgmap v73: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T21:00:00.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:00 smithi096 ceph-mon[29288]: overall HEALTH_OK 2024-04-25T21:00:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:00 smithi097 ceph-mon[32667]: pgmap v73: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T21:00:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:00 smithi097 ceph-mon[32667]: overall HEALTH_OK 2024-04-25T21:00:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:00 smithi139 ceph-mon[33027]: pgmap v73: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T21:00:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:00 smithi139 ceph-mon[33027]: overall HEALTH_OK 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 -- 172.21.15.96:0/441709245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4102a90 msgr2=0x7fade4102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 --2- 172.21.15.96:0/441709245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4102a90 0x7fade4102f10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fadd0009990 tx=0x7fadd0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 -- 172.21.15.96:0/441709245 shutdown_connections 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 --2- 172.21.15.96:0/441709245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4103450 0x7fade410f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 --2- 172.21.15.96:0/441709245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fade4108a50 0x7fade4108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 --2- 172.21.15.96:0/441709245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4102a90 0x7fade4102f10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 -- 172.21.15.96:0/441709245 >> 172.21.15.96:0/441709245 conn(0x7fade4079ba0 msgr2=0x7fade4079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 -- 172.21.15.96:0/441709245 shutdown_connections 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 -- 172.21.15.96:0/441709245 wait complete. 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.208+0000 7fadead72700 1 Processor -- start 2024-04-25T21:00:02.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 -- start start 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4102a90 0x7fade4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 0x7fade4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fade4108a50 0x7fade419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112c00 con 0x7fade4103450 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112a80 con 0x7fade4102a90 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fadead72700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fade4112d80 con 0x7fade4108a50 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 0x7fade4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 0x7fade4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35858/0 (socket says 172.21.15.96:35858) 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 -- 172.21.15.96:0/2256296729 learned_addr learned my addr 172.21.15.96:0/2256296729 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4102a90 0x7fade4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade930f700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fade4108a50 0x7fade419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 -- 172.21.15.96:0/2256296729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fade4108a50 msgr2=0x7fade419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fade4108a50 0x7fade419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.212 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 -- 172.21.15.96:0/2256296729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4102a90 msgr2=0x7fade4199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4102a90 0x7fade4199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 -- 172.21.15.96:0/2256296729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade419e140 con 0x7fade4103450 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.209+0000 7fade3fff700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 0x7fade4199670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000c00 tx=0x7fadd00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.210+0000 7fade1ffb700 1 -- 172.21.15.96:0/2256296729 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0027070 con 0x7fade4103450 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.210+0000 7fade1ffb700 1 -- 172.21.15.96:0/2256296729 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd00040e0 con 0x7fade4103450 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.210+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade419e3d0 con 0x7fade4103450 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.210+0000 7fade1ffb700 1 -- 172.21.15.96:0/2256296729 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd000f460 con 0x7fade4103450 2024-04-25T21:00:02.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.210+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade41a5cb0 con 0x7fade4103450 2024-04-25T21:00:02.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.212+0000 7fade1ffb700 1 -- 172.21.15.96:0/2256296729 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fadd002a070 con 0x7fade4103450 2024-04-25T21:00:02.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.213+0000 7fade1ffb700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fadd40a3e30 0x7fadd40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.213+0000 7fade1ffb700 1 -- 172.21.15.96:0/2256296729 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fadd00d16e0 con 0x7fade4103450 2024-04-25T21:00:02.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.213+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fadd40a3e30 0x7fadd40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.213+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fadd40a3e30 0x7fadd40a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005950 tx=0x7fadd800a300 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.213+0000 7fadf55fbb80 1 Processor -- start 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 -- start start 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e963a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 0x55614e972d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e973260 0x55614e975650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55614e8b6d70 con 0x55614e963f50 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55614e85da10 con 0x55614e973260 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55614e8b2d30 con 0x55614e963630 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 0x55614e972d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 0x55614e972d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35860/0 (socket says 172.21.15.96:35860) 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 -- 172.21.15.96:0/1966608296 learned_addr learned my addr 172.21.15.96:0/1966608296 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:02.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade930f700 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e973260 0x55614e975650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade8b0e700 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e963a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 -- 172.21.15.96:0/1966608296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 msgr2=0x55614e963a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e963a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 -- 172.21.15.96:0/1966608296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e973260 msgr2=0x55614e975650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e973260 0x55614e975650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 -- 172.21.15.96:0/1966608296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614e821820 con 0x55614e963f50 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fade3fff700 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 0x55614e972d20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fadd0099be0 tx=0x7fadd009d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb7ca0b19879bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadceffd700 1 -- 172.21.15.96:0/1966608296 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd009d780 con 0x55614e963f50 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadceffd700 1 -- 172.21.15.96:0/1966608296 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd00a1b30 con 0x55614e963f50 2024-04-25T21:00:02.217 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.214+0000 7fadceffd700 1 -- 172.21.15.96:0/1966608296 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd009d9e0 con 0x55614e963f50 2024-04-25T21:00:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 -- 172.21.15.96:0/1966608296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 msgr2=0x55614e972d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.221 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 0x55614e972d20 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fadd0099be0 tx=0x7fadd009d2c0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 -- 172.21.15.96:0/1966608296 shutdown_connections 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e973260 0x55614e975650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e963a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/1966608296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e963f50 0x55614e972d20 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 -- 172.21.15.96:0/1966608296 >> 172.21.15.96:0/1966608296 conn(0x55614e890970 msgr2=0x55614e95faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 -- 172.21.15.96:0/1966608296 shutdown_connections 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 -- 172.21.15.96:0/1966608296 wait complete. 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.215+0000 7fadf55fbb80 1 -- start start 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 Processor -- start 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e972a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963f50 0x55614e96fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.222 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e970290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55614e8b6d70 con 0x55614e973260 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55614e870690 con 0x55614e963f50 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55614e975b90 con 0x55614e963630 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fadf55fbb80 10 client.0 _ll_register_callbacks cb 0x55614e84fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e970290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e970290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35876/0 (socket says 172.21.15.96:35876) 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade3fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963f50 0x55614e96fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 -- 172.21.15.96:0/2904425819 learned_addr learned my addr 172.21.15.96:0/2904425819 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e972a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 -- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 msgr2=0x55614e972a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e972a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 -- 172.21.15.96:0/2904425819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963f50 msgr2=0x55614e96fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963f50 0x55614e96fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade930f700 1 -- 172.21.15.96:0/2904425819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614e8049b0 con 0x55614e973260 2024-04-25T21:00:02.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e972a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.216+0000 7fade3fff700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963f50 0x55614e96fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fade930f700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e970290 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faddc008f20 tx=0x7faddc010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ea77511cd51d75d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadc3fff700 1 -- 172.21.15.96:0/2904425819 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc010da0 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadc3fff700 1 -- 172.21.15.96:0/2904425819 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7faddc016470 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fadd00a1b30 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadc3fff700 1 -- 172.21.15.96:0/2904425819 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faddc024050 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadf55fbb80 10 client.15585 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55614e980320 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadc3fff700 1 -- 172.21.15.96:0/2904425819 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd00a1b30 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadc3fff700 1 -- 172.21.15.96:0/2904425819 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faddc015560 con 0x55614e973260 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadf55fbb80 10 client.15585 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadf55fbb80 10 client.15585 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.217+0000 7fadf55fbb80 20 client.15585 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:02.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.218+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9829b0 0x55614e984da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.218+0000 7fadf55fbb80 4 client.15585 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.218+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55614e8c4560 con 0x55614e9829b0 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.218+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9829b0 0x55614e984da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.218+0000 7fadc3fff700 10 client.15585 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.218+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9829b0 0x55614e984da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4caa181d3383249e server_cookie=4ed142feddc73228 in_seq=0 out_seq=0 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadc3fff700 1 -- 172.21.15.96:0/2904425819 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55614e8c4560 con 0x55614e9829b0 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadc3fff700 10 client.15585 handle_command_reply: tid=0 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 client.15585 shutdown 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 msgr2=0x55614e970290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e970290 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faddc008f20 tx=0x7faddc010430 comp rx=0 tx=0).stop 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 shutdown_connections 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e970290 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.225 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963630 0x55614e972a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9829b0 0x55614e984da0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadc3fff700 0 client.15585 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2904425819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963f50 0x55614e96fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 >> 172.21.15.96:0/2904425819 conn(0x55614e890970 msgr2=0x55614e965fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 shutdown_connections 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2904425819 wait complete. 2024-04-25T21:00:02.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.219+0000 7fadf55fbb80 20 client.15585 trim_cache size 0 max 16384 2024-04-25T21:00:02.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 Processor -- start 2024-04-25T21:00:02.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 -- start start 2024-04-25T21:00:02.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e965e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e98a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e98abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ad820 con 0x55614e973260 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fade41ab0b0 con 0x55614e963630 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a8f40 con 0x55614e963f50 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e98abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e98abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35878/0 (socket says 172.21.15.96:35878) 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 -- 172.21.15.96:0/2627309504 learned_addr learned my addr 172.21.15.96:0/2627309504 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade8b0e700 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e965e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade3fff700 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e98a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 -- 172.21.15.96:0/2627309504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 msgr2=0x55614e98a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e98a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 -- 172.21.15.96:0/2627309504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 msgr2=0x55614e965e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e965e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 -- 172.21.15.96:0/2627309504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55614e95f3a0 con 0x55614e973260 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.237+0000 7fade930f700 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e98abb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faddc015950 tx=0x7faddc0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bd56f0023aa78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadce7fc700 1 -- 172.21.15.96:0/2627309504 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc02c020 con 0x55614e973260 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadce7fc700 1 -- 172.21.15.96:0/2627309504 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faddc01b070 con 0x55614e973260 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadce7fc700 1 -- 172.21.15.96:0/2627309504 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc004d80 con 0x55614e973260 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2627309504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 msgr2=0x55614e98abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e98abb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faddc015950 tx=0x7faddc0078c0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2627309504 shutdown_connections 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e98abb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e965e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/2627309504 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e98a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2627309504 >> 172.21.15.96:0/2627309504 conn(0x55614e890970 msgr2=0x55614e965740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2627309504 shutdown_connections 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.238+0000 7fadf55fbb80 1 -- 172.21.15.96:0/2627309504 wait complete. 2024-04-25T21:00:02.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 -- start start 2024-04-25T21:00:02.242 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 Processor -- start 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e976a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e976f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e9774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a9590 con 0x55614e973260 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fade41aa9e0 con 0x55614e963630 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fade41a9c40 con 0x55614e963f50 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fadf55fbb80 10 client.0 _ll_register_callbacks cb 0x55614e84fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fade930f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e9774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fade930f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e9774c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35890/0 (socket says 172.21.15.96:35890) 2024-04-25T21:00:02.245 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fade930f700 1 -- 172.21.15.96:0/3789841123 learned_addr learned my addr 172.21.15.96:0/3789841123 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.239+0000 7fade8b0e700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e976a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade3fff700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e976f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade930f700 1 -- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 msgr2=0x55614e976f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade930f700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e976f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade930f700 1 -- 172.21.15.96:0/3789841123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 msgr2=0x55614e976a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade930f700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e976a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade930f700 1 -- 172.21.15.96:0/3789841123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55614e9e8140 con 0x55614e973260 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fade930f700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e9774c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faddc007400 tx=0x7faddc008840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dc1f41739f9aa01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadc3fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadc3fff700 1 -- 172.21.15.96:0/3789841123 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faddc02c040 con 0x55614e973260 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faddc01b070 con 0x55614e973260 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadc3fff700 1 -- 172.21.15.96:0/3789841123 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7faddc0153d0 con 0x55614e973260 2024-04-25T21:00:02.246 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadc3fff700 1 -- 172.21.15.96:0/3789841123 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7faddc0155d0 con 0x55614e973260 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadf55fbb80 10 client.15591 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55614e9e9420 con 0x55614e973260 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadc3fff700 1 -- 172.21.15.96:0/3789841123 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faddc01b070 con 0x55614e973260 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.240+0000 7fadc3fff700 1 -- 172.21.15.96:0/3789841123 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7faddc004020 con 0x55614e973260 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadf55fbb80 10 client.15591 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadf55fbb80 10 client.15591 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadf55fbb80 20 client.15591 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9e9a20 0x55614e9ebe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadf55fbb80 4 client.15591 mds_command: new command op to 24442 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-2c77974d-1-mds.i.json"}] 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 --> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-2c77974d-1-mds.i.json"}) -- 0x55614e8be600 con 0x55614e9e9a20 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fade8b0e700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9e9a20 0x55614e9ebe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fadc3fff700 10 client.15591 ms_handle_connect on v2:172.21.15.139:6838/2573809872 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.241+0000 7fade8b0e700 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9e9a20 0x55614e9ebe10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f08e81727578f4 server_cookie=2c75521c62ef02a4 in_seq=0 out_seq=0 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.242+0000 7fadc3fff700 1 -- 172.21.15.96:0/3789841123 <== mds.1 v2:172.21.15.139:6838/2573809872 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x55614e8be600 con 0x55614e9e9a20 2024-04-25T21:00:02.247 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.242+0000 7fadc3fff700 10 client.15591 handle_command_reply: tid=0 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 client.15591 shutdown 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 msgr2=0x55614e9774c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e9774c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7faddc007400 tx=0x7faddc008840 comp rx=0 tx=0).stop 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 shutdown_connections 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55614e973260 0x55614e9774c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:6838/2573809872,v1:172.21.15.139:6839/2573809872] conn(0x55614e9e9a20 0x55614e9ebe10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55614e963630 0x55614e976a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 --2- 172.21.15.96:0/3789841123 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55614e963f50 0x55614e976f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadc3fff700 0 client.15591 ms_handle_reset on v2:172.21.15.139:6838/2573809872 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 >> 172.21.15.96:0/3789841123 conn(0x55614e890970 msgr2=0x55614e95ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 shutdown_connections 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 1 -- 172.21.15.96:0/3789841123 wait complete. 2024-04-25T21:00:02.248 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadf55fbb80 20 client.15591 trim_cache size 0 max 16384 2024-04-25T21:00:02.249 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:00:02.249 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/var/run/ceph/ops-2c77974d-1-mds.i.json" 2024-04-25T21:00:02.249 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:00:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fadd40a3e30 msgr2=0x7fadd40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.249 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadead72700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fadd40a3e30 0x7fadd40a62f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005950 tx=0x7fadd800a300 comp rx=0 tx=0).stop 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 msgr2=0x7fade4199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.243+0000 7fadead72700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 0x7fade4199670 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fadd0000c00 tx=0x7fadd00047b0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 shutdown_connections 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fadd40a3e30 0x7fadd40a62f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fade4108a50 0x7fade419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fade4102a90 0x7fade4199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 --2- 172.21.15.96:0/2256296729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fade4103450 0x7fade4199670 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 >> 172.21.15.96:0/2256296729 conn(0x7fade4079ba0 msgr2=0x7fade41018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 shutdown_connections 2024-04-25T21:00:02.250 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:02.244+0000 7fadead72700 1 -- 172.21.15.96:0/2256296729 wait complete. 2024-04-25T21:00:02.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:02 smithi096 ceph-mon[29288]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T21:00:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:02 smithi097 ceph-mon[32667]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T21:00:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:02 smithi139 ceph-mon[33027]: pgmap v74: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.5 KiB/s wr, 7 op/s 2024-04-25T21:00:03.055 DEBUG:tasks.quiescer.fs.[cephfs]:Dumping ops on rank 0 (b) to a remote file /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.b.json 2024-04-25T21:00:03.055 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph tell mds.1:0 ops --flags=locks --path=/var/run/ceph/ops-2c77974d-1-mds.b.json 2024-04-25T21:00:03.293 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:04.550 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:04 smithi096 ceph-mon[29288]: pgmap v75: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T21:00:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:04 smithi097 ceph-mon[32667]: pgmap v75: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T21:00:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:04 smithi139 ceph-mon[33027]: pgmap v75: 97 pgs: 97 active+clean; 20 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 -- 172.21.15.96:0/1490467213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 msgr2=0x7f684c10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 --2- 172.21.15.96:0/1490467213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c10f8e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f684400b0a0 tx=0x7f684400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 -- 172.21.15.96:0/1490467213 shutdown_connections 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 --2- 172.21.15.96:0/1490467213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c10f8e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 --2- 172.21.15.96:0/1490467213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 0x7f684c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 --2- 172.21.15.96:0/1490467213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f684c102b00 0x7f684c102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 -- 172.21.15.96:0/1490467213 >> 172.21.15.96:0/1490467213 conn(0x7f684c079c00 msgr2=0x7f684c07a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:04.962 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 -- 172.21.15.96:0/1490467213 shutdown_connections 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.959+0000 7f6851bd3700 1 -- 172.21.15.96:0/1490467213 wait complete. 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 Processor -- start 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 -- start start 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f684c102b00 0x7f684c199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 0x7f684c19dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f684c112cb0 con 0x7f684c1034c0 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f684c112b30 con 0x7f684c108a10 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f6851bd3700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f684c112e30 con 0x7f684c102b00 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f684affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f684affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35904/0 (socket says 172.21.15.96:35904) 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.960+0000 7f684affd700 1 -- 172.21.15.96:0/3029586100 learned_addr learned my addr 172.21.15.96:0/3029586100 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f684c102b00 0x7f684c199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.963 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684bfff700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 0x7f684c19dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684affd700 1 -- 172.21.15.96:0/3029586100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f684c102b00 msgr2=0x7f684c199200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684affd700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f684c102b00 0x7f684c199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684affd700 1 -- 172.21.15.96:0/3029586100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 msgr2=0x7f684c19dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684affd700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 0x7f684c19dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684affd700 1 -- 172.21.15.96:0/3029586100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f684c19e210 con 0x7f684c1034c0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684bfff700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 0x7f684c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f684affd700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c199740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f684000b530 tx=0x7f684000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f6848ff9700 1 -- 172.21.15.96:0/3029586100 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840015020 con 0x7f684c1034c0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f6848ff9700 1 -- 172.21.15.96:0/3029586100 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6840011ce0 con 0x7f684c1034c0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f6848ff9700 1 -- 172.21.15.96:0/3029586100 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840010d20 con 0x7f684c1034c0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f684c19e500 con 0x7f684c1034c0 2024-04-25T21:00:04.964 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.961+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f684c1a5d80 con 0x7f684c1034c0 2024-04-25T21:00:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.963+0000 7f6848ff9700 1 -- 172.21.15.96:0/3029586100 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6840018020 con 0x7f684c1034c0 2024-04-25T21:00:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.964+0000 7f6848ff9700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f68340a3db0 0x7f68340a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.964+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f68340a3db0 0x7f68340a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.964+0000 7f6848ff9700 1 -- 172.21.15.96:0/3029586100 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f68400cfed0 con 0x7f684c1034c0 2024-04-25T21:00:04.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.964+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f68340a3db0 0x7f68340a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c008040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.964+0000 7f685c45cb80 1 Processor -- start 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 -- start start 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 0x562930cdca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930ce7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930cea240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562930c2fd70 con 0x562930cdc630 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562930bd6a10 con 0x562930cdcf50 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f685c45cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562930c2bd30 con 0x562930ce7e50 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 0x562930cdca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 0x562930cdca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35920/0 (socket says 172.21.15.96:35920) 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 -- 172.21.15.96:0/1856955498 learned_addr learned my addr 172.21.15.96:0/1856955498 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684affd700 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930ce7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 -- 172.21.15.96:0/1856955498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 msgr2=0x562930cea240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:04.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684bfff700 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930cea240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930cea240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 -- 172.21.15.96:0/1856955498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 msgr2=0x562930ce7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930ce7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 -- 172.21.15.96:0/1856955498 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562930b9a820 con 0x562930cdc630 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.965+0000 7f684b7fe700 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 0x562930cdca10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f683c013970 tx=0x7f683c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74de5b8e912222d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f6831ffb700 1 -- 172.21.15.96:0/1856955498 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c019040 con 0x562930cdc630 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f6831ffb700 1 -- 172.21.15.96:0/1856955498 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f683c022070 con 0x562930cdc630 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f6831ffb700 1 -- 172.21.15.96:0/1856955498 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c01dab0 con 0x562930cdc630 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 -- 172.21.15.96:0/1856955498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 msgr2=0x562930cdca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 0x562930cdca10 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f683c013970 tx=0x7f683c00f000 comp rx=0 tx=0).stop 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 -- 172.21.15.96:0/1856955498 shutdown_connections 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930cea240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdc630 0x562930cdca10 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 --2- 172.21.15.96:0/1856955498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930ce7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 -- 172.21.15.96:0/1856955498 >> 172.21.15.96:0/1856955498 conn(0x562930c09970 msgr2=0x562930cd8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 -- 172.21.15.96:0/1856955498 shutdown_connections 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.966+0000 7f685c45cb80 1 -- 172.21.15.96:0/1856955498 wait complete. 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 -- start start 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 Processor -- start 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdc630 0x562930ce7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930ce4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930ce4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562930ceecf0 con 0x562930cdcf50 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x562930be9690 con 0x562930cdc630 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x562930ceee70 con 0x562930ce7e50 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f685c45cb80 10 client.0 _ll_register_callbacks cb 0x562930bc8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f684affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930ce4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f684affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930ce4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:35932/0 (socket says 172.21.15.96:35932) 2024-04-25T21:00:04.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f684affd700 1 -- 172.21.15.96:0/3939551130 learned_addr learned my addr 172.21.15.96:0/3939551130 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.967+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdc630 0x562930ce7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684bfff700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930ce4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684affd700 1 -- 172.21.15.96:0/3939551130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 msgr2=0x562930ce4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684affd700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930ce4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684affd700 1 -- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdc630 msgr2=0x562930ce7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684affd700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdc630 0x562930ce7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684affd700 1 -- 172.21.15.96:0/3939551130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562930b7d9b0 con 0x562930cdcf50 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdc630 0x562930ce7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f684affd700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930ce4900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f68400cd800 tx=0x7f684009c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44aeea80dc90d7b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f682affd700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f682affd700 1 -- 172.21.15.96:0/3939551130 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68400ba060 con 0x562930cdcf50 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f682affd700 1 -- 172.21.15.96:0/3939551130 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f684009c450 con 0x562930cdcf50 2024-04-25T21:00:04.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f683c022070 con 0x562930cdcf50 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f682affd700 1 -- 172.21.15.96:0/3939551130 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68400b5070 con 0x562930cdcf50 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f685c45cb80 10 client.15603 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562930cf9630 con 0x562930cdcf50 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.968+0000 7f682affd700 1 -- 172.21.15.96:0/3939551130 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f683c022070 con 0x562930cdcf50 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f682affd700 1 -- 172.21.15.96:0/3939551130 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f684009b4d0 con 0x562930cdcf50 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f685c45cb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f685c45cb80 10 client.15603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f685c45cb80 20 client.15603 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f685c45cb80 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930cfbff0 0x562930cfe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f685c45cb80 4 client.15603 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562930c3d560 con 0x562930cfbff0 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930cfbff0 0x562930cfe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f682affd700 10 client.15603 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:04.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:04.969+0000 7f684b7fe700 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930cfbff0 0x562930cfe430 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62f6824b3fefb86f server_cookie=aece57d39e995e22 in_seq=0 out_seq=0 2024-04-25T21:00:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:06 smithi097 ceph-mon[32667]: pgmap v76: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-25T21:00:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:06 smithi139 ceph-mon[33027]: pgmap v76: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-25T21:00:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:06 smithi096 ceph-mon[29288]: pgmap v76: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 23 op/s 2024-04-25T21:00:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:08 smithi097 ceph-mon[32667]: pgmap v77: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:08 smithi139 ceph-mon[33027]: pgmap v77: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:08 smithi096 ceph-mon[29288]: pgmap v77: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:10 smithi097 ceph-mon[32667]: pgmap v78: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:10 smithi139 ceph-mon[33027]: pgmap v78: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:10 smithi096 ceph-mon[29288]: pgmap v78: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe747fff640 1 -- 172.21.15.96:0/4077887102 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5612f3e29e90 con 0x5612f3ea3300 2024-04-25T21:00:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe747fff640 10 client.15549 handle_command_reply: tid=0 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 client.15549 shutdown 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 msgr2=0x5612f3e263f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e263f0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fe75000a690 tx=0x7fe75000af30 comp rx=0 tx=0).stop 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 shutdown_connections 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5612f3ea3300 0x5612f3ea56f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612f3e2ca10 0x5612f3e263f0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5612f3e1fda0 0x5612f3e25eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe747fff640 0 client.15549 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 --2- 172.21.15.96:0/4077887102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5612f3e1f480 0x5612f3e34c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 >> 172.21.15.96:0/4077887102 conn(0x5612f3e1bb30 msgr2=0x5612f3e24d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 shutdown_connections 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 1 -- 172.21.15.96:0/4077887102 wait complete. 2024-04-25T21:00:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.167+0000 7fe771ba6740 20 client.15549 trim_cache size 0 max 16384 2024-04-25T21:00:12.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.167+0000 7f682affd700 1 -- 172.21.15.96:0/3939551130 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562930c3d560 con 0x562930cfbff0 2024-04-25T21:00:12.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.167+0000 7f682affd700 10 client.15603 handle_command_reply: tid=0 2024-04-25T21:00:12.170 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.167+0000 7f685c45cb80 1 client.15603 shutdown 2024-04-25T21:00:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:00:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:00:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:00:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:00:12.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 msgr2=0x562930ce4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.171 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930ce4900 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f68400cd800 tx=0x7f684009c7a0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 shutdown_connections 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930cfbff0 0x562930cfe430 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930ce7e50 0x562930ce4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdc630 0x562930ce7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f682affd700 0 client.15603 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 --2- 172.21.15.96:0/3939551130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930ce4900 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 >> 172.21.15.96:0/3939551130 conn(0x562930c09970 msgr2=0x562930cdefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 shutdown_connections 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 1 -- 172.21.15.96:0/3939551130 wait complete. 2024-04-25T21:00:12.172 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.168+0000 7f685c45cb80 20 client.15603 trim_cache size 0 max 16384 2024-04-25T21:00:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe7540a3dc0 msgr2=0x7fe7540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe7540a3dc0 0x7fe7540a6280 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe7580eeb20 tx=0x7fe75808e7b0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 msgr2=0x7fe76819bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 0x7fe76819bec0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fe758009a20 tx=0x7fe758004270 comp rx=0 tx=0).stop 2024-04-25T21:00:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 shutdown_connections 2024-04-25T21:00:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fe7540a3dc0 0x7fe7540a6280 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe768106fb0 0x7fe76819bec0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe768101b60 0x7fe768197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 --2- 172.21.15.96:0/3644807963 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fe7681011a0 0x7fe768197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 >> 172.21.15.96:0/3644807963 conn(0x7fe768076c50 msgr2=0x7fe7680ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.168+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 shutdown_connections 2024-04-25T21:00:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:12.169+0000 7fe76edad640 1 -- 172.21.15.96:0/3644807963 wait complete. 2024-04-25T21:00:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 Processor -- start 2024-04-25T21:00:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 -- start start 2024-04-25T21:00:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930d03d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930ce7e50 0x562930d04280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.187 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1ad9b0 con 0x562930cdcf50 2024-04-25T21:00:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1ab240 con 0x562930ce7e50 2024-04-25T21:00:12.188 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f685c45cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1a9090 con 0x562930cdc630 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930d03d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930d03d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46004/0 (socket says 172.21.15.96:46004) 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 -- 172.21.15.96:0/673558077 learned_addr learned my addr 172.21.15.96:0/673558077 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684bfff700 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930ce7e50 0x562930d04280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684b7fe700 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 -- 172.21.15.96:0/673558077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 msgr2=0x562930cf7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 -- 172.21.15.96:0/673558077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930ce7e50 msgr2=0x562930d04280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930ce7e50 0x562930d04280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 -- 172.21.15.96:0/673558077 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562930cde090 con 0x562930cdcf50 2024-04-25T21:00:12.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684b7fe700 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f684affd700 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930d03d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f68400989c0 tx=0x7f68400cf000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07347d37f16bfd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.187+0000 7f68317fa700 1 -- 172.21.15.96:0/673558077 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68400a4c60 con 0x562930cdcf50 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f68317fa700 1 -- 172.21.15.96:0/673558077 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684009bff0 con 0x562930cdcf50 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f68317fa700 1 -- 172.21.15.96:0/673558077 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68400a47d0 con 0x562930cdcf50 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 -- 172.21.15.96:0/673558077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 msgr2=0x562930d03d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930d03d40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f68400989c0 tx=0x7f68400cf000 comp rx=0 tx=0).stop 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 -- 172.21.15.96:0/673558077 shutdown_connections 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930ce7e50 0x562930d04280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 --2- 172.21.15.96:0/673558077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930cdcf50 0x562930d03d40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 -- 172.21.15.96:0/673558077 >> 172.21.15.96:0/673558077 conn(0x562930c09970 msgr2=0x562930cd7c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 -- 172.21.15.96:0/673558077 shutdown_connections 2024-04-25T21:00:12.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.188+0000 7f685c45cb80 1 -- 172.21.15.96:0/673558077 wait complete. 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 -- start start 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 Processor -- start 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930cf2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 0x562930d615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1a9700 con 0x562930ce7e50 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1aab70 con 0x562930cdcf50 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f684c1a9dd0 con 0x562930cdc630 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f685c45cb80 10 client.0 _ll_register_callbacks cb 0x562930bc8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f684bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 0x562930d615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f684affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930cf2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f684bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 0x562930d615c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46010/0 (socket says 172.21.15.96:46010) 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.189+0000 7f684bfff700 1 -- 172.21.15.96:0/4269635795 learned_addr learned my addr 172.21.15.96:0/4269635795 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684b7fe700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684affd700 1 -- 172.21.15.96:0/4269635795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 msgr2=0x562930cf22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684affd700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf22d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684affd700 1 -- 172.21.15.96:0/4269635795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 msgr2=0x562930d615c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684affd700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 0x562930d615c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684affd700 1 -- 172.21.15.96:0/4269635795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562930d61bb0 con 0x562930cdcf50 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684bfff700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 0x562930d615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f684affd700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930cf2810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f68400989c0 tx=0x7f68400b56a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aced4e0d6b8c36f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:12.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f682affd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f682affd700 1 -- 172.21.15.96:0/4269635795 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840010ec0 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f684009bff0 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f682affd700 1 -- 172.21.15.96:0/4269635795 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f684009c250 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f682affd700 1 -- 172.21.15.96:0/4269635795 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f68400a4c60 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f685c45cb80 10 client.25079 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562930d61bb0 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.190+0000 7f682affd700 1 -- 172.21.15.96:0/4269635795 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f684009bff0 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f682affd700 1 -- 172.21.15.96:0/4269635795 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7f684009aeb0 con 0x562930cdcf50 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f685c45cb80 10 client.25079 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f685c45cb80 10 client.25079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f685c45cb80 20 client.25079 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f685c45cb80 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930d63180 0x562930d655a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:12.196 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f685c45cb80 4 client.25079 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-2c77974d-1-mds.b.json"}] 2024-04-25T21:00:12.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "ops", "flags": ["locks"], "path": "/var/run/ceph/ops-2c77974d-1-mds.b.json"}) -- 0x562930c37600 con 0x562930d63180 2024-04-25T21:00:12.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f684b7fe700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930d63180 0x562930d655a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:12.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f684b7fe700 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930d63180 0x562930d655a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42ade97b44c0b98c server_cookie=6f0f5899f4c824ae in_seq=0 out_seq=0 2024-04-25T21:00:12.197 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.191+0000 7f682affd700 10 client.25079 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:12.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f682affd700 1 -- 172.21.15.96:0/4269635795 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+58 (crc 0 0 0) 0x562930c37600 con 0x562930d63180 2024-04-25T21:00:12.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f682affd700 10 client.25079 handle_command_reply: tid=0 2024-04-25T21:00:12.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 client.25079 shutdown 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 msgr2=0x562930cf2810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930cf2810 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f68400989c0 tx=0x7f68400b56a0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 shutdown_connections 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562930ce7e50 0x562930d615c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x562930d63180 0x562930d655a0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x562930cdc630 0x562930cf22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 --2- 172.21.15.96:0/4269635795 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x562930cdcf50 0x562930cf2810 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f682affd700 0 client.25079 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 >> 172.21.15.96:0/4269635795 conn(0x562930c09970 msgr2=0x562930cf6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 shutdown_connections 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 1 -- 172.21.15.96:0/4269635795 wait complete. 2024-04-25T21:00:12.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.268+0000 7f685c45cb80 20 client.25079 trim_cache size 0 max 16384 2024-04-25T21:00:12.270 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:00:12.270 INFO:teuthology.orchestra.run.smithi096.stdout: "path": "/var/run/ceph/ops-2c77974d-1-mds.b.json" 2024-04-25T21:00:12.270 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f68340a3db0 msgr2=0x7f68340a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f68340a3db0 0x7f68340a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f683c006fd0 tx=0x7f683c008040 comp rx=0 tx=0).stop 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 msgr2=0x7f684c199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c199740 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f684000b530 tx=0x7f684000b8f0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 shutdown_connections 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f68340a3db0 0x7f68340a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f684c108a10 0x7f684c19dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f684c102b00 0x7f684c199200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 --2- 172.21.15.96:0/3029586100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f684c1034c0 0x7f684c199740 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:12.271 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 >> 172.21.15.96:0/3029586100 conn(0x7f684c079c00 msgr2=0x7f684c100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 shutdown_connections 2024-04-25T21:00:12.272 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:12.269+0000 7f6851bd3700 1 -- 172.21.15.96:0/3029586100 wait complete. 2024-04-25T21:00:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:12 smithi096 ceph-mon[29288]: pgmap v79: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T21:00:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:12 smithi097 ceph-mon[32667]: pgmap v79: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T21:00:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:12 smithi139 ceph-mon[33027]: pgmap v79: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-04-25T21:00:13.029 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-04-25T21:00:13.043 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.Y8Tjinl6cg 2024-04-25T21:00:13.044 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.h.json /tmp/tmp.Y8Tjinl6cg 2024-04-25T21:00:13.110 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.Y8Tjinl6cg 2024-04-25T21:00:13.233 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.Y8Tjinl6cg is 24KB 2024-04-25T21:00:13.282 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.Y8Tjinl6cg 2024-04-25T21:00:13.296 DEBUG:tasks.quiescer.fs.[cephfs]:read 24349B of ops from '/var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.h.json' on mds.2 (h) 2024-04-25T21:00:13.300 INFO:tasks.quiescer.fs.[cephfs]:Pulled 14 ops from rank 2 (h) into home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/ops-2c77974d-1-mds.h.json 2024-04-25T21:00:13.300 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -- /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.h.json 2024-04-25T21:00:13.359 DEBUG:teuthology.orchestra.run.smithi139:> mktemp 2024-04-25T21:00:13.378 INFO:teuthology.orchestra.run.smithi139.stdout:/tmp/tmp.czzkFDrlqb 2024-04-25T21:00:13.378 DEBUG:teuthology.orchestra.run.smithi139:> sudo cp /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.i.json /tmp/tmp.czzkFDrlqb 2024-04-25T21:00:13.449 DEBUG:teuthology.orchestra.run.smithi139:> sudo chmod 0666 /tmp/tmp.czzkFDrlqb 2024-04-25T21:00:13.575 DEBUG:teuthology.orchestra.remote:smithi139:/tmp/tmp.czzkFDrlqb is 24KB 2024-04-25T21:00:13.597 DEBUG:teuthology.orchestra.run.smithi139:> rm -fr /tmp/tmp.czzkFDrlqb 2024-04-25T21:00:13.611 DEBUG:tasks.quiescer.fs.[cephfs]:read 24338B of ops from '/var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.i.json' on mds.1 (i) 2024-04-25T21:00:13.615 INFO:tasks.quiescer.fs.[cephfs]:Pulled 14 ops from rank 1 (i) into home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/ops-2c77974d-1-mds.i.json 2024-04-25T21:00:13.615 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -- /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.i.json 2024-04-25T21:00:13.675 DEBUG:teuthology.orchestra.run.smithi097:> mktemp 2024-04-25T21:00:13.689 INFO:teuthology.orchestra.run.smithi097.stdout:/tmp/tmp.UsTilfaEe9 2024-04-25T21:00:13.690 DEBUG:teuthology.orchestra.run.smithi097:> sudo cp /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.b.json /tmp/tmp.UsTilfaEe9 2024-04-25T21:00:13.755 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 0666 /tmp/tmp.UsTilfaEe9 2024-04-25T21:00:13.876 DEBUG:teuthology.orchestra.remote:smithi097:/tmp/tmp.UsTilfaEe9 is 2MB 2024-04-25T21:00:13.998 DEBUG:teuthology.orchestra.run.smithi097:> rm -fr /tmp/tmp.UsTilfaEe9 2024-04-25T21:00:14.015 DEBUG:tasks.quiescer.fs.[cephfs]:read 2561018B of ops from '/var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.b.json' on mds.0 (b) 2024-04-25T21:00:14.080 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:954 create owner_uid=1000, owner_gid=1316 #0x100000003a4/086 2024-04-25T21:00:12.023087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.082 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:986 create owner_uid=1000, owner_gid=1316 #0x100000003a4/094 2024-04-25T21:00:12.047086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.084 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1018 create owner_uid=1000, owner_gid=1316 #0x100000003a4/102 2024-04-25T21:00:12.071086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.085 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1050 create owner_uid=1000, owner_gid=1316 #0x100000003a4/110 2024-04-25T21:00:12.097085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.087 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1082 create owner_uid=1000, owner_gid=1316 #0x100000003a4/118 2024-04-25T21:00:12.132084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.089 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1114 create owner_uid=1000, owner_gid=1316 #0x100000003a4/126 2024-04-25T21:00:12.167084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.091 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1146 create owner_uid=1000, owner_gid=1316 #0x100000003a4/134 2024-04-25T21:00:12.187083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.093 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:956 create owner_uid=1000, owner_gid=1316 #0x100000003a4/086.out 2024-04-25T21:00:12.024087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.095 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:988 create owner_uid=1000, owner_gid=1316 #0x100000003a4/094.out 2024-04-25T21:00:12.049086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.096 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1020 create owner_uid=1000, owner_gid=1316 #0x100000003a4/102.out 2024-04-25T21:00:12.073086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.098 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1052 create owner_uid=1000, owner_gid=1316 #0x100000003a4/110.out 2024-04-25T21:00:12.099085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.100 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1084 create owner_uid=1000, owner_gid=1316 #0x100000003a4/118.out 2024-04-25T21:00:12.134084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.102 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1116 create owner_uid=1000, owner_gid=1316 #0x100000003a4/126.out 2024-04-25T21:00:12.168084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.103 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1148 create owner_uid=1000, owner_gid=1316 #0x100000003a4/134.out 2024-04-25T21:00:12.188083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.105 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:958 create owner_uid=1000, owner_gid=1316 #0x100000003a4/087 2024-04-25T21:00:12.026087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.107 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:990 create owner_uid=1000, owner_gid=1316 #0x100000003a4/095 2024-04-25T21:00:12.051086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.109 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1022 create owner_uid=1000, owner_gid=1316 #0x100000003a4/103 2024-04-25T21:00:12.074086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.111 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1054 create owner_uid=1000, owner_gid=1316 #0x100000003a4/111 2024-04-25T21:00:12.100085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.113 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1086 create owner_uid=1000, owner_gid=1316 #0x100000003a4/119 2024-04-25T21:00:12.137084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.115 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1118 create owner_uid=1000, owner_gid=1316 #0x100000003a4/127 2024-04-25T21:00:12.169084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.117 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1150 create owner_uid=1000, owner_gid=1316 #0x100000003a4/135 2024-04-25T21:00:12.189083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.119 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:960 create owner_uid=1000, owner_gid=1316 #0x100000003a4/087.out 2024-04-25T21:00:12.027087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.121 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:992 create owner_uid=1000, owner_gid=1316 #0x100000003a4/095.out 2024-04-25T21:00:12.052086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.123 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1024 create owner_uid=1000, owner_gid=1316 #0x100000003a4/103.out 2024-04-25T21:00:12.076086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.125 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1056 create owner_uid=1000, owner_gid=1316 #0x100000003a4/111.out 2024-04-25T21:00:12.101085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.127 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1088 create owner_uid=1000, owner_gid=1316 #0x100000003a4/119.out 2024-04-25T21:00:12.139084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.129 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1120 create owner_uid=1000, owner_gid=1316 #0x100000003a4/127.out 2024-04-25T21:00:12.171084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.131 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1152 create owner_uid=1000, owner_gid=1316 #0x100000003a4/135.out 2024-04-25T21:00:12.191083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.133 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:962 create owner_uid=1000, owner_gid=1316 #0x100000003a4/088 2024-04-25T21:00:12.030087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.135 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:994 create owner_uid=1000, owner_gid=1316 #0x100000003a4/096 2024-04-25T21:00:12.053086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.137 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1026 create owner_uid=1000, owner_gid=1316 #0x100000003a4/104 2024-04-25T21:00:12.077086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.139 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1058 create owner_uid=1000, owner_gid=1316 #0x100000003a4/112 2024-04-25T21:00:12.104085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.141 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1090 create owner_uid=1000, owner_gid=1316 #0x100000003a4/120 2024-04-25T21:00:12.141084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.143 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1122 create owner_uid=1000, owner_gid=1316 #0x100000003a4/128 2024-04-25T21:00:12.172084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.145 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:964 create owner_uid=1000, owner_gid=1316 #0x100000003a4/088.out 2024-04-25T21:00:12.031087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.147 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:996 create owner_uid=1000, owner_gid=1316 #0x100000003a4/096.out 2024-04-25T21:00:12.055086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.149 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1028 create owner_uid=1000, owner_gid=1316 #0x100000003a4/104.out 2024-04-25T21:00:12.080086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.151 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1060 create owner_uid=1000, owner_gid=1316 #0x100000003a4/112.out 2024-04-25T21:00:12.104085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.152 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1092 create owner_uid=1000, owner_gid=1316 #0x100000003a4/120.out 2024-04-25T21:00:12.144084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.154 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1124 create owner_uid=1000, owner_gid=1316 #0x100000003a4/128.out 2024-04-25T21:00:12.173084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.156 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:966 create owner_uid=1000, owner_gid=1316 #0x100000003a4/089 2024-04-25T21:00:12.032087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.158 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:998 create owner_uid=1000, owner_gid=1316 #0x100000003a4/097 2024-04-25T21:00:12.057086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.160 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1030 create owner_uid=1000, owner_gid=1316 #0x100000003a4/105 2024-04-25T21:00:12.081086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.161 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1062 create owner_uid=1000, owner_gid=1316 #0x100000003a4/113 2024-04-25T21:00:12.107085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.163 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1094 create owner_uid=1000, owner_gid=1316 #0x100000003a4/121 2024-04-25T21:00:12.146084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.165 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1126 create owner_uid=1000, owner_gid=1316 #0x100000003a4/129 2024-04-25T21:00:12.174084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.167 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:968 create owner_uid=1000, owner_gid=1316 #0x100000003a4/089.out 2024-04-25T21:00:12.034087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.168 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1000 create owner_uid=1000, owner_gid=1316 #0x100000003a4/097.out 2024-04-25T21:00:12.057086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.170 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1032 create owner_uid=1000, owner_gid=1316 #0x100000003a4/105.out 2024-04-25T21:00:12.082086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.172 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1064 create owner_uid=1000, owner_gid=1316 #0x100000003a4/113.out 2024-04-25T21:00:12.109085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.173 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1096 create owner_uid=1000, owner_gid=1316 #0x100000003a4/121.out 2024-04-25T21:00:12.148084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.175 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1128 create owner_uid=1000, owner_gid=1316 #0x100000003a4/129.out 2024-04-25T21:00:12.176083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.177 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:938 create owner_uid=1000, owner_gid=1316 #0x100000003a4/082 2024-04-25T21:00:12.010087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.179 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:970 create owner_uid=1000, owner_gid=1316 #0x100000003a4/090 2024-04-25T21:00:12.035087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.180 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1002 create owner_uid=1000, owner_gid=1316 #0x100000003a4/098 2024-04-25T21:00:12.060086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.182 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1034 create owner_uid=1000, owner_gid=1316 #0x100000003a4/106 2024-04-25T21:00:12.084086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.184 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1066 create owner_uid=1000, owner_gid=1316 #0x100000003a4/114 2024-04-25T21:00:12.111085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.185 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1098 create owner_uid=1000, owner_gid=1316 #0x100000003a4/122 2024-04-25T21:00:12.151084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.187 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1130 create owner_uid=1000, owner_gid=1316 #0x100000003a4/130 2024-04-25T21:00:12.177083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.189 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:940 create owner_uid=1000, owner_gid=1316 #0x100000003a4/082.out 2024-04-25T21:00:12.012087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.190 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:972 create owner_uid=1000, owner_gid=1316 #0x100000003a4/090.out 2024-04-25T21:00:12.037087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.192 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1004 create owner_uid=1000, owner_gid=1316 #0x100000003a4/098.out 2024-04-25T21:00:12.061086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.193 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1036 create owner_uid=1000, owner_gid=1316 #0x100000003a4/106.out 2024-04-25T21:00:12.085085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.195 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1068 create owner_uid=1000, owner_gid=1316 #0x100000003a4/114.out 2024-04-25T21:00:12.114085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.197 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1100 create owner_uid=1000, owner_gid=1316 #0x100000003a4/122.out 2024-04-25T21:00:12.152084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.199 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1132 create owner_uid=1000, owner_gid=1316 #0x100000003a4/130.out 2024-04-25T21:00:12.178083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.200 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:942 create owner_uid=1000, owner_gid=1316 #0x100000003a4/083 2024-04-25T21:00:12.014087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.202 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:974 create owner_uid=1000, owner_gid=1316 #0x100000003a4/091 2024-04-25T21:00:12.038087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.204 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1006 create owner_uid=1000, owner_gid=1316 #0x100000003a4/099 2024-04-25T21:00:12.063086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.206 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1038 create owner_uid=1000, owner_gid=1316 #0x100000003a4/107 2024-04-25T21:00:12.088085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.207 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1070 create owner_uid=1000, owner_gid=1316 #0x100000003a4/115 2024-04-25T21:00:12.116085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.209 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1102 create owner_uid=1000, owner_gid=1316 #0x100000003a4/123 2024-04-25T21:00:12.155084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.211 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1134 create owner_uid=1000, owner_gid=1316 #0x100000003a4/131 2024-04-25T21:00:12.180083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.213 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:944 create owner_uid=1000, owner_gid=1316 #0x100000003a4/083.out 2024-04-25T21:00:12.016087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.215 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:976 create owner_uid=1000, owner_gid=1316 #0x100000003a4/091.out 2024-04-25T21:00:12.040086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.216 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1008 create owner_uid=1000, owner_gid=1316 #0x100000003a4/099.out 2024-04-25T21:00:12.064086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.218 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1040 create owner_uid=1000, owner_gid=1316 #0x100000003a4/107.out 2024-04-25T21:00:12.089085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.220 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1072 create owner_uid=1000, owner_gid=1316 #0x100000003a4/115.out 2024-04-25T21:00:12.118085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.222 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1104 create owner_uid=1000, owner_gid=1316 #0x100000003a4/123.out 2024-04-25T21:00:12.158084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.223 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1136 create owner_uid=1000, owner_gid=1316 #0x100000003a4/131.out 2024-04-25T21:00:12.181083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.225 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:946 create owner_uid=1000, owner_gid=1316 #0x100000003a4/084 2024-04-25T21:00:12.017087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.227 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:978 create owner_uid=1000, owner_gid=1316 #0x100000003a4/092 2024-04-25T21:00:12.041086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.228 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1010 create owner_uid=1000, owner_gid=1316 #0x100000003a4/100 2024-04-25T21:00:12.065086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.230 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1042 create owner_uid=1000, owner_gid=1316 #0x100000003a4/108 2024-04-25T21:00:12.090085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.231 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1074 create owner_uid=1000, owner_gid=1316 #0x100000003a4/116 2024-04-25T21:00:12.121085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.233 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1106 create owner_uid=1000, owner_gid=1316 #0x100000003a4/124 2024-04-25T21:00:12.160084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.235 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1138 create owner_uid=1000, owner_gid=1316 #0x100000003a4/132 2024-04-25T21:00:12.182083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.237 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:948 create owner_uid=1000, owner_gid=1316 #0x100000003a4/084.out 2024-04-25T21:00:12.018087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.239 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:980 create owner_uid=1000, owner_gid=1316 #0x100000003a4/092.out 2024-04-25T21:00:12.043086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.240 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1012 create owner_uid=1000, owner_gid=1316 #0x100000003a4/100.out 2024-04-25T21:00:12.068086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.242 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1044 create owner_uid=1000, owner_gid=1316 #0x100000003a4/108.out 2024-04-25T21:00:12.092085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.244 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1076 create owner_uid=1000, owner_gid=1316 #0x100000003a4/116.out 2024-04-25T21:00:12.125085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.246 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1108 create owner_uid=1000, owner_gid=1316 #0x100000003a4/124.out 2024-04-25T21:00:12.162084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.248 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1140 create owner_uid=1000, owner_gid=1316 #0x100000003a4/132.out 2024-04-25T21:00:12.183083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.249 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:950 create owner_uid=1000, owner_gid=1316 #0x100000003a4/085 2024-04-25T21:00:12.020087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.251 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:982 create owner_uid=1000, owner_gid=1316 #0x100000003a4/093 2024-04-25T21:00:12.045086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.253 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1014 create owner_uid=1000, owner_gid=1316 #0x100000003a4/101 2024-04-25T21:00:12.068086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.255 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1046 create owner_uid=1000, owner_gid=1316 #0x100000003a4/109 2024-04-25T21:00:12.094085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.256 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1078 create owner_uid=1000, owner_gid=1316 #0x100000003a4/117 2024-04-25T21:00:12.127085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.258 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1110 create owner_uid=1000, owner_gid=1316 #0x100000003a4/125 2024-04-25T21:00:12.164084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.260 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1142 create owner_uid=1000, owner_gid=1316 #0x100000003a4/133 2024-04-25T21:00:12.184083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.262 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:952 create owner_uid=1000, owner_gid=1316 #0x100000003a4/085.out 2024-04-25T21:00:12.021087+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.264 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:984 create owner_uid=1000, owner_gid=1316 #0x100000003a4/093.out 2024-04-25T21:00:12.046086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.265 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1016 create owner_uid=1000, owner_gid=1316 #0x100000003a4/101.out 2024-04-25T21:00:12.070086+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.267 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1048 create owner_uid=1000, owner_gid=1316 #0x100000003a4/109.out 2024-04-25T21:00:12.096085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.269 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1080 create owner_uid=1000, owner_gid=1316 #0x100000003a4/117.out 2024-04-25T21:00:12.130085+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.271 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1112 create owner_uid=1000, owner_gid=1316 #0x100000003a4/125.out 2024-04-25T21:00:12.165084+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.272 DEBUG:tasks.quiescer.fs.[cephfs]:Outstanding op at rank 0 (b) for 2c77974d-1: 'client_request(client.24731:1144 create owner_uid=1000, owner_gid=1316 #0x100000003a4/133.out 2024-04-25T21:00:12.187083+0000 caller_uid=1000, caller_gid=1316{6,36,1000,1316,})' 2024-04-25T21:00:14.277 INFO:tasks.quiescer.fs.[cephfs]:Pulled 108 ops from rank 0 (b) into home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/ops-2c77974d-1-mds.b.json 2024-04-25T21:00:14.277 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -- /var/run/ceph/b713778a-0344-11ef-bc93-c7b262605968/ops-2c77974d-1-mds.b.json 2024-04-25T21:00:14.306 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:00:14.555 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:14.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:14 smithi097 ceph-mon[32667]: pgmap v80: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:14 smithi139 ceph-mon[33027]: pgmap v80: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:14.678 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:14 smithi096 ceph-mon[29288]: pgmap v80: 97 pgs: 97 active+clean; 20 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 20 op/s 2024-04-25T21:00:16.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 -- 172.21.15.96:0/618036582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 msgr2=0x7f2e9c102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:16.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/618036582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c102f80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c009990 tx=0x7f2e8c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 -- 172.21.15.96:0/618036582 shutdown_connections 2024-04-25T21:00:16.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/618036582 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e9c1034c0 0x7f2e9c10f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.199 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/618036582 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 0x7f2e9c108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/618036582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c102f80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 -- 172.21.15.96:0/618036582 >> 172.21.15.96:0/618036582 conn(0x7f2e9c079c00 msgr2=0x7f2e9c07a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 -- 172.21.15.96:0/618036582 shutdown_connections 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.196+0000 7f2ea23fc700 1 -- 172.21.15.96:0/618036582 wait complete. 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 Processor -- start 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 -- start start 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e9c1034c0 0x7f2e9c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 0x7f2e9c19da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c112c50 con 0x7f2e9c102b00 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c112ad0 con 0x7f2e9c1034c0 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2ea23fc700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2e9c112dd0 con 0x7f2e9c108a10 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2e9bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c1991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46040/0 (socket says 172.21.15.96:46040) 2024-04-25T21:00:16.200 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2e9bfff700 1 -- 172.21.15.96:0/3967600348 learned_addr learned my addr 172.21.15.96:0/3967600348 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.197+0000 7f2e9b7fe700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e9c1034c0 0x7f2e9c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2ea0999700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 0x7f2e9c19da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e9bfff700 1 -- 172.21.15.96:0/3967600348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 msgr2=0x7f2e9c19da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e9bfff700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 0x7f2e9c19da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e9bfff700 1 -- 172.21.15.96:0/3967600348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e9c1034c0 msgr2=0x7f2e9c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e9bfff700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e9c1034c0 0x7f2e9c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e9bfff700 1 -- 172.21.15.96:0/3967600348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9c19e1b0 con 0x7f2e9c102b00 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2ea0999700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 0x7f2e9c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e9bfff700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c1991a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b530 tx=0x7f2e9000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90015020 con 0x7f2e9c102b00 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e90011ce0 con 0x7f2e9c102b00 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9c19e4a0 con 0x7f2e9c102b00 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90010e40 con 0x7f2e9c102b00 2024-04-25T21:00:16.201 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.198+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e9c1a5d20 con 0x7f2e9c102b00 2024-04-25T21:00:16.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.199+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e9c104150 con 0x7f2e9c102b00 2024-04-25T21:00:16.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.201+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2e90018020 con 0x7f2e9c102b00 2024-04-25T21:00:16.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.201+0000 7f2e997fa700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2e840a3e30 0x7f2e840a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:16.202 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.201+0000 7f2e9b7fe700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2e840a3e30 0x7f2e840a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:16.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.201+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7f2e900d0440 con 0x7f2e9c102b00 2024-04-25T21:00:16.203 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.202+0000 7f2e9b7fe700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2e840a3e30 0x7f2e840a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098e0 tx=0x7f2e8c005960 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:16.206 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.205+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e9001f020 con 0x7f2e9c102b00 2024-04-25T21:00:16.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.448+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e9c19a500 con 0x7f2e9c102b00 2024-04-25T21:00:16.449 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.449+0000 7f2e997fa700 1 -- 172.21.15.96:0/3967600348 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11862 (secure 0 0 0) 0x7f2e9009cc60 con 0x7f2e9c102b00 2024-04-25T21:00:16.450 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:00:16.450 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T20:59:13.489761+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/57160731","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":57160731},{"type":"v1","addr":"172.21.15.139:6837","nonce":57160731}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2e840a3e30 msgr2=0x7f2e840a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2e840a3e30 0x7f2e840a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2e8c0098e0 tx=0x7f2e8c005960 comp rx=0 tx=0).stop 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 msgr2=0x7f2e9c1991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c1991a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000b530 tx=0x7f2e9000b840 comp rx=0 tx=0).stop 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 shutdown_connections 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2e840a3e30 0x7f2e840a62f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2e9c108a10 0x7f2e9c19da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9c102b00 0x7f2e9c1991a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 --2- 172.21.15.96:0/3967600348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e9c1034c0 0x7f2e9c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 >> 172.21.15.96:0/3967600348 conn(0x7f2e9c079c00 msgr2=0x7f2e9c100cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:16.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 shutdown_connections 2024-04-25T21:00:16.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:16.451+0000 7f2ea23fc700 1 -- 172.21.15.96:0/3967600348 wait complete. 2024-04-25T21:00:16.453 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 27 2024-04-25T21:00:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:16 smithi097 ceph-mon[32667]: pgmap v81: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 391 KiB/s wr, 54 op/s 2024-04-25T21:00:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:16 smithi139 ceph-mon[33027]: pgmap v81: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 391 KiB/s wr, 54 op/s 2024-04-25T21:00:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:16 smithi096 ceph-mon[29288]: pgmap v81: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 391 KiB/s wr, 54 op/s 2024-04-25T21:00:17.212 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24443 quiesce db --set-id 2c77974d --await-for 42.0' 2024-04-25T21:00:17.212 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24443 quiesce db --set-id 2c77974d --await-for 42.0 2024-04-25T21:00:17.492 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:00:17.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:17 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3967600348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:17.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:17 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3967600348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:17.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:17 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3967600348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:00:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:18 smithi097 ceph-mon[32667]: pgmap v82: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 367 KiB/s wr, 36 op/s 2024-04-25T21:00:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:18 smithi139 ceph-mon[33027]: pgmap v82: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 367 KiB/s wr, 36 op/s 2024-04-25T21:00:18.675 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:18 smithi096 ceph-mon[29288]: pgmap v82: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 367 KiB/s wr, 36 op/s 2024-04-25T21:00:19.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.226+0000 7fc2b703f700 1 -- 172.21.15.96:0/775736758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b006b1f0 msgr2=0x7fc2b006b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.229 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.226+0000 7fc2b703f700 1 --2- 172.21.15.96:0/775736758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b006b1f0 0x7fc2b006b670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc29c009990 tx=0x7fc29c009ca0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 -- 172.21.15.96:0/775736758 shutdown_connections 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 --2- 172.21.15.96:0/775736758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b006bbb0 0x7fc2b010f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 --2- 172.21.15.96:0/775736758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b0108a60 0x7fc2b0108e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 --2- 172.21.15.96:0/775736758 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b006b1f0 0x7fc2b006b670 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 -- 172.21.15.96:0/775736758 >> 172.21.15.96:0/775736758 conn(0x7fc2b00789e0 msgr2=0x7fc2b0078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 -- 172.21.15.96:0/775736758 shutdown_connections 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 -- 172.21.15.96:0/775736758 wait complete. 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 Processor -- start 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 -- start start 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b006b1f0 0x7fc2b01990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.227+0000 7fc2b703f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b006bbb0 0x7fc2b0199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b703f700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b0108a60 0x7fc2b019d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b703f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0112ca0 con 0x7fc2b006b1f0 2024-04-25T21:00:19.230 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b703f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0112b20 con 0x7fc2b006bbb0 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b703f700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b0112e20 con 0x7fc2b0108a60 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b006b1f0 0x7fc2b01990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b006b1f0 0x7fc2b01990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54132/0 (socket says 172.21.15.96:54132) 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/4091921439 learned_addr learned my addr 172.21.15.96:0/4091921439 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2affff700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b006bbb0 0x7fc2b0199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b0108a60 0x7fc2b019d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/4091921439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b0108a60 msgr2=0x7fc2b019d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc2b0108a60 0x7fc2b019d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/4091921439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b006bbb0 msgr2=0x7fc2b0199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc2b006bbb0 0x7fc2b0199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/4091921439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b019e100 con 0x7fc2b006b1f0 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b006b1f0 0x7fc2b01990f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400eb30 tx=0x7fc2a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:19.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2adffb700 1 -- 172.21.15.96:0/4091921439 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400cbd0 con 0x7fc2b006b1f0 2024-04-25T21:00:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2b703f700 1 -- 172.21.15.96:0/4091921439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b019e3f0 con 0x7fc2b006b1f0 2024-04-25T21:00:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2adffb700 1 -- 172.21.15.96:0/4091921439 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a40044d0 con 0x7fc2b006b1f0 2024-04-25T21:00:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.228+0000 7fc2adffb700 1 -- 172.21.15.96:0/4091921439 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4003df0 con 0x7fc2b006b1f0 2024-04-25T21:00:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.229+0000 7fc2b703f700 1 -- 172.21.15.96:0/4091921439 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b01a5c70 con 0x7fc2b006b1f0 2024-04-25T21:00:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.231+0000 7fc2adffb700 1 -- 172.21.15.96:0/4091921439 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2a4004000 con 0x7fc2b006b1f0 2024-04-25T21:00:19.232 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.231+0000 7fc2adffb700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc2a00a3e30 0x7fc2a00a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.231+0000 7fc2affff700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc2a00a3e30 0x7fc2a00a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.231+0000 7fc2adffb700 1 -- 172.21.15.96:0/4091921439 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fc2a40d0980 con 0x7fc2b006b1f0 2024-04-25T21:00:19.233 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.232+0000 7fc2affff700 1 --2- 172.21.15.96:0/4091921439 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc2a00a3e30 0x7fc2a00a62f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc29c00b2e0 tx=0x7fc29c005960 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:19.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.232+0000 7fc2c18c8b80 1 Processor -- start 2024-04-25T21:00:19.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.232+0000 7fc2c18c8b80 1 -- start start 2024-04-25T21:00:19.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 0x5598adb14dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.234 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb1fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb201e0 0x5598adb225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5598ada67e30 con 0x5598adb149f0 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598ada0ea60 con 0x5598adb201e0 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5598ada63df0 con 0x5598adb15310 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 0x5598adb14dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b55dc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb201e0 0x5598adb225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 0x5598adb14dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54138/0 (socket says 172.21.15.96:54138) 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2172395923 learned_addr learned my addr 172.21.15.96:0/2172395923 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2affff700 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb1fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2172395923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 msgr2=0x5598adb1fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb1fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2172395923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb201e0 msgr2=0x5598adb225d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.235 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb201e0 0x5598adb225d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2172395923 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598ad9b59d0 con 0x5598adb149f0 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2affff700 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb1fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb201e0 0x5598adb225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 0x5598adb14dd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a4098c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25910a3614273f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc29affd700 1 -- 172.21.15.96:0/2172395923 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a409c290 con 0x5598adb149f0 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc29affd700 1 -- 172.21.15.96:0/2172395923 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a4009500 con 0x5598adb149f0 2024-04-25T21:00:19.236 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.233+0000 7fc29affd700 1 -- 172.21.15.96:0/2172395923 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40a55a0 con 0x5598adb149f0 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2172395923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 msgr2=0x5598adb14dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 0x5598adb14dd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009fd0 tx=0x7fc2a4098c50 comp rx=0 tx=0).stop 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2172395923 shutdown_connections 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb201e0 0x5598adb225d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb149f0 0x5598adb14dd0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2172395923 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb1fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2172395923 >> 172.21.15.96:0/2172395923 conn(0x5598ada42790 msgr2=0x5598adb10ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2172395923 shutdown_connections 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.234+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2172395923 wait complete. 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 -- start start 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 Processor -- start 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb149f0 0x5598adb1f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.239 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb15310 0x5598adb1cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb1d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5598adb22bd0 con 0x5598adb201e0 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5598ada06cd0 con 0x5598adb15310 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5598adb22d50 con 0x5598adb149f0 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2c18c8b80 10 client.0 _ll_register_callbacks cb 0x5598ada00a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2b55dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb1d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2b55dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb1d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54146/0 (socket says 172.21.15.96:54146) 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2b55dc700 1 -- 172.21.15.96:0/3769085246 learned_addr learned my addr 172.21.15.96:0/3769085246 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2affff700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb15310 0x5598adb1cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.235+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb149f0 0x5598adb1f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b55dc700 1 -- 172.21.15.96:0/3769085246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb149f0 msgr2=0x5598adb1f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb149f0 0x5598adb1f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b55dc700 1 -- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb15310 msgr2=0x5598adb1cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.240 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb15310 0x5598adb1cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b55dc700 1 -- 172.21.15.96:0/3769085246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598ada1f370 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb149f0 0x5598adb1f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2affff700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb15310 0x5598adb1cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb1d180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8011e20 tx=0x7fc2a8014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce53b96a7c6bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc28ffff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc28ffff700 1 -- 172.21.15.96:0/3769085246 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a801e420 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc28ffff700 1 -- 172.21.15.96:0/3769085246 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fc2a801e5c0 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a4009500 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc28ffff700 1 -- 172.21.15.96:0/3769085246 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2a801e7c0 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2c18c8b80 10 client.15627 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598adb2d480 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc28ffff700 1 -- 172.21.15.96:0/3769085246 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a4009500 con 0x5598adb201e0 2024-04-25T21:00:19.241 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.236+0000 7fc28ffff700 1 -- 172.21.15.96:0/3769085246 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc2a8026030 con 0x5598adb201e0 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2c18c8b80 10 client.15627 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2c18c8b80 10 client.15627 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2c18c8b80 20 client.15627 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb31f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2c18c8b80 4 client.15627 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598ada75620 con 0x5598adb2fb10 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb31f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc28ffff700 10 client.15627 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:19.243 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.237+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb31f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e28e7418653fda server_cookie=cf55a301c12e4818 in_seq=0 out_seq=0 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc28ffff700 1 -- 172.21.15.96:0/3769085246 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5598ada75620 con 0x5598adb2fb10 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc28ffff700 10 client.15627 handle_command_reply: tid=0 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 client.15627 shutdown 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 msgr2=0x5598adb1d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb1d180 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8011e20 tx=0x7fc2a8014600 comp rx=0 tx=0).stop 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 shutdown_connections 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb1d180 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb31f00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc28ffff700 0 client.15627 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.238+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb149f0 0x5598adb1f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.239+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/3769085246 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb15310 0x5598adb1cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.239+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 >> 172.21.15.96:0/3769085246 conn(0x5598ada42790 msgr2=0x5598adb10b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.239+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 shutdown_connections 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.239+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/3769085246 wait complete. 2024-04-25T21:00:19.244 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.239+0000 7fc2c18c8b80 20 client.15627 trim_cache size 0 max 16384 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 Processor -- start 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 -- start start 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb377b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb37cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb19f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01afb10 con 0x5598adb201e0 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ad3a0 con 0x5598adb149f0 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ab210 con 0x5598adb15310 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2b55dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb19f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2b55dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb19f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54154/0 (socket says 172.21.15.96:54154) 2024-04-25T21:00:19.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2b55dc700 1 -- 172.21.15.96:0/1325238382 learned_addr learned my addr 172.21.15.96:0/1325238382 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:19.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.258+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb377b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b55dc700 1 -- 172.21.15.96:0/1325238382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 msgr2=0x5598adb37cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2affff700 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb37cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb37cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b55dc700 1 -- 172.21.15.96:0/1325238382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 msgr2=0x5598adb377b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb377b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b55dc700 1 -- 172.21.15.96:0/1325238382 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598adb32510 con 0x5598adb201e0 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb377b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb19f00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8011310 tx=0x7fc2a8011340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecff76083b41fb8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:19.263 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc29a7fc700 1 -- 172.21.15.96:0/1325238382 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a801e620 con 0x5598adb201e0 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc29a7fc700 1 -- 172.21.15.96:0/1325238382 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a8016400 con 0x5598adb201e0 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.259+0000 7fc29a7fc700 1 -- 172.21.15.96:0/1325238382 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a801ed60 con 0x5598adb201e0 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/1325238382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 msgr2=0x5598adb19f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb19f00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8011310 tx=0x7fc2a8011340 comp rx=0 tx=0).stop 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/1325238382 shutdown_connections 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb201e0 0x5598adb19f00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb377b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/1325238382 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb15310 0x5598adb37cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/1325238382 >> 172.21.15.96:0/1325238382 conn(0x5598ada42790 msgr2=0x5598adb1b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/1325238382 shutdown_connections 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/1325238382 wait complete. 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 -- start start 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.260+0000 7fc2c18c8b80 1 Processor -- start 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb93660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb15310 0x5598adb93ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.264 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb201e0 0x5598adb8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01ab860 con 0x5598adb15310 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01accd0 con 0x5598adb149f0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b01abf30 con 0x5598adb201e0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 10 client.0 _ll_register_callbacks cb 0x5598ada00a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb93660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb93660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:44300/0 (socket says 172.21.15.96:44300) 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2228787405 learned_addr learned my addr 172.21.15.96:0/2228787405 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b55dc700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb201e0 0x5598adb8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2affff700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb15310 0x5598adb93ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2228787405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb201e0 msgr2=0x5598adb8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb201e0 0x5598adb8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2228787405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb15310 msgr2=0x5598adb93ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb15310 0x5598adb93ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 -- 172.21.15.96:0/2228787405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598adb8e6e0 con 0x5598adb149f0 2024-04-25T21:00:19.265 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2affff700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb15310 0x5598adb93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2b4ddb700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb93660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009230 tx=0x7fc2a4003890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20a7ad4abe25dbd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc28ffff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc28ffff700 1 -- 172.21.15.96:0/2228787405 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a409d030 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a8016400 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.261+0000 7fc28ffff700 1 -- 172.21.15.96:0/2228787405 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7fc2a4098fa0 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc28ffff700 1 -- 172.21.15.96:0/2228787405 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc2a40a55a0 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 10 client.25103 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5598adb9ae30 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc28ffff700 1 -- 172.21.15.96:0/2228787405 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2a8016400 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc28ffff700 1 -- 172.21.15.96:0/2228787405 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7fc2a4083020 con 0x5598adb149f0 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 10 client.25103 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:19.266 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 10 client.25103 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 20 client.25103 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 4 client.25103 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2c77974d", "await_for": 42.0}] 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2c77974d", "await_for": 42.0}) -- 0x5598ada6f6c0 con 0x5598adb2fb10 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.262+0000 7fc2affff700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc28ffff700 10 client.25103 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:19.267 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2affff700 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb9d070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf2d2a509fe1250 server_cookie=9f6ecd2d490806bc in_seq=0 out_seq=0 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc28ffff700 1 -- 172.21.15.96:0/2228787405 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: -110 ) ==== 8+0+590 (crc 0 0 0) 0x5598ada6f6c0 con 0x5598adb2fb10 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc28ffff700 10 client.25103 handle_command_reply: tid=0 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 client.25103 shutdown 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 msgr2=0x5598adb93660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb93660 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4009230 tx=0x7fc2a4003890 comp rx=0 tx=0).stop 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 shutdown_connections 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5598adb2fb10 0x5598adb9d070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.268 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5598adb201e0 0x5598adb8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5598adb149f0 0x5598adb93660 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 --2- 172.21.15.96:0/2228787405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5598adb15310 0x5598adb93ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc28ffff700 0 client.25103 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 >> 172.21.15.96:0/2228787405 conn(0x5598ada42790 msgr2=0x5598adb1e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 shutdown_connections 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 1 -- 172.21.15.96:0/2228787405 wait complete. 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:00:19.263+0000 7fc2c18c8b80 20 client.25103 trim_cache size 0 max 16384 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 27, 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "leader": 24443, 2024-04-25T21:00:19.269 INFO:teuthology.orchestra.run.smithi096.stdout: "set_version": 6, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "sets": { 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "2c77974d": { 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "version": 6, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age_ref": 7.5, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "TIMEDOUT", 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 0.0 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "timeout": 60.0, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "expiration": 86.2, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "members": { 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "file:/": { 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "excluded": false, 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "state": { 2024-04-25T21:00:19.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "QUIESCING", 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: "age": 60.0 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:00:19.271 INFO:teuthology.orchestra.run.smithi096.stderr:Error ETIMEDOUT: 2024-04-25T21:00:20.128 DEBUG:teuthology.orchestra.run:got remote process result: 110 2024-04-25T21:00:20.129 ERROR:tasks.quiescer.fs.[cephfs]:Couldn't quiesce root with rc: 110 (ETIMEDOUT), stdout: { "epoch": 27, "leader": 24443, "set_version": 6, "sets": { "2c77974d": { "version": 6, "age_ref": 7.5, "state": { "name": "TIMEDOUT", "age": 0.0 }, "timeout": 60.0, "expiration": 86.2, "members": { "file:/": { "excluded": false, "state": { "name": "QUIESCING", "age": 60.0 } } } } } } 2024-04-25T21:00:20.129 ERROR:tasks.quiescer.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 353, in _run self.do_quiesce(d) File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 308, in do_quiesce raise RuntimeError(f"Error quiescing set '{set_id}': {rcinfo}") RuntimeError: Error quiescing set '2c77974d': 110 (ETIMEDOUT) 2024-04-25T21:00:20.486 INFO:tasks.workunit.client.0.smithi096.stderr:Updating files: 11% (491/4290) Updating files: 12% (515/4290) Updating files: 13% (558/4290) Updating files: 14% (601/4290) Updating files: 14% (634/4290) Updating files: 15% (644/4290) Updating files: 16% (687/4290) Updating files: 17% (730/4290) Updating files: 18% (773/4290) Updating files: 19% (816/4290) Updating files: 20% (858/4290) Updating files: 21% (901/4290) Updating files: 22% (944/4290) Updating files: 23% (987/4290) Updating files: 24% (1030/4290) Updating files: 25% (1073/4290) Updating files: 26% (1116/4290) Updating files: 27% (1159/4290) Updating files: 28% (1202/4290) Updating files: 29% (1245/4290) Updating files: 30% (1287/4290) Updating files: 30% (1288/4290) Updating files: 31% (1330/4290) Updating files: 32% (1373/4290) Updating files: 33% (1416/4290) Updating files: 34% (1459/4290) Updating files: 35% (1502/4290) Updating files: 36% (1545/4290) Updating files: 37% (1588/4290) Updating files: 38% (1631/4290) Updating files: 39% (1674/4290) Updating files: 40% (1716/4290) Updating files: 40% (1735/4290) Updating files: 41% (1759/4290) Updating files: 42% (1802/4290) Updating files: 43% (1845/4290) Updating files: 44% (1888/4290) Updating files: 45% (1931/4290) Updating files: 46% (1974/4290) Updating files: 47% (2017/4290) Updating files: 48% (2060/4290) Updating files: 49% (2103/4290) Updating files: 50% (2145/4290) Updating files: 51% (2188/4290) Updating files: 52% (2231/4290) Updating files: 53% (2274/4290) Updating files: 53% (2315/4290) Updating files: 54% (2317/4290) Updating files: 55% (2360/4290) Updating files: 56% (2403/4290) Updating files: 57% (2446/4290) Updating files: 58% (2489/4290) Updating files: 59% (2532/4290) Updating files: 60% (2574/4290) Updating files: 61% (2617/4290) Updating files: 62% (2660/4290) Updating files: 63% (2703/4290) Updating files: 64% (2746/4290) Updating files: 64% (2784/4290) Updating files: 65% (2789/4290) Updating files: 66% (2832/4290) Updating files: 67% (2875/4290) Updating files: 68% (2918/4290) Updating files: 69% (2961/4290) Updating files: 70% (3003/4290) Updating files: 70% (3022/4290) Updating files: 71% (3046/4290) Updating files: 72% (3089/4290) Updating files: 73% (3132/4290) Updating files: 74% (3175/4290) Updating files: 75% (3218/4290) Updating files: 76% (3261/4290) Updating files: 77% (3304/4290) Updating files: 78% (3347/4290) Updating files: 79% (3390/4290) Updating files: 80% (3432/4290) Updating files: 81% (3475/4290) Updating files: 82% (3518/4290) Updating files: 83% (3561/4290) Updating files: 84% (3604/4290) Updating files: 85% (3647/4290) Updating files: 85% (3677/4290) Updating files: 86% (3690/4290) Updating files: 87% (3733/4290) Updating files: 88% (3776/4290) Updating files: 89% (3819/4290) Updating files: 90% (3861/4290) Updating files: 91% (3904/4290) Updating files: 92% (3947/4290) Updating files: 93% (3990/4290) Updating files: 94% (4033/4290) Updating files: 95% (4076/4290) Updating files: 96% (4119/4290) Updating files: 97% (4162/4290) Updating files: 97% (4185/4290) Updating files: 98% (4205/4290) Updating files: 99% (4248/4290) Updating files: 100% (4290/4290) Updating files: 100% (4290/4290), done. 2024-04-25T21:00:20.522 INFO:tasks.workunit.client.0.smithi096.stderr:+ cd xfstests-dev 2024-04-25T21:00:20.522 INFO:tasks.workunit.client.0.smithi096.stderr:+ make -j4 2024-04-25T21:00:20.531 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize -cfi 2024-04-25T21:00:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:20 smithi097 ceph-mon[32667]: pgmap v83: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 367 KiB/s wr, 36 op/s 2024-04-25T21:00:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:20 smithi139 ceph-mon[33027]: pgmap v83: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 367 KiB/s wr, 36 op/s 2024-04-25T21:00:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:20 smithi096 ceph-mon[29288]: pgmap v83: 97 pgs: 97 active+clean; 24 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 367 KiB/s wr, 36 op/s 2024-04-25T21:00:20.892 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: putting auxiliary files in '.'. 2024-04-25T21:00:20.893 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './config.guess' 2024-04-25T21:00:20.897 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './config.sub' 2024-04-25T21:00:20.902 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './install-sh' 2024-04-25T21:00:20.907 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file './ltmain.sh' 2024-04-25T21:00:20.934 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-04-25T21:00:20.934 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-04-25T21:00:20.940 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-04-25T21:00:20.945 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-04-25T21:00:20.951 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-04-25T21:00:20.956 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-04-25T21:00:20.962 INFO:tasks.workunit.client.0.smithi096.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-04-25T21:00:20.963 INFO:tasks.workunit.client.0.smithi096.stdout:cp include/install-sh . 2024-04-25T21:00:20.977 INFO:tasks.workunit.client.0.smithi096.stdout:aclocal -I m4 2024-04-25T21:00:21.998 INFO:tasks.workunit.client.0.smithi096.stdout:autoheader 2024-04-25T21:00:22.305 INFO:tasks.workunit.client.0.smithi096.stdout:autoconf 2024-04-25T21:00:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:22 smithi097 ceph-mon[32667]: pgmap v84: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 827 KiB/s wr, 58 op/s 2024-04-25T21:00:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:22 smithi139 ceph-mon[33027]: pgmap v84: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 827 KiB/s wr, 58 op/s 2024-04-25T21:00:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:22 smithi096 ceph-mon[29288]: pgmap v84: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 827 KiB/s wr, 58 op/s 2024-04-25T21:00:24.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:24 smithi097 ceph-mon[32667]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 827 KiB/s wr, 56 op/s 2024-04-25T21:00:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:24 smithi139 ceph-mon[33027]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 827 KiB/s wr, 56 op/s 2024-04-25T21:00:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:24 smithi096 ceph-mon[29288]: pgmap v85: 97 pgs: 97 active+clean; 28 MiB data, 473 MiB used, 1.0 TiB / 1.0 TiB avail; 542 KiB/s rd, 827 KiB/s wr, 56 op/s 2024-04-25T21:00:25.092 INFO:tasks.workunit.client.0.smithi096.stdout:./configure \ 2024-04-25T21:00:25.092 INFO:tasks.workunit.client.0.smithi096.stdout: --libexecdir=/usr/lib \ 2024-04-25T21:00:25.092 INFO:tasks.workunit.client.0.smithi096.stdout: --exec_prefix=/var/lib 2024-04-25T21:00:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/2564676547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e0107270 msgr2=0x7ff2e010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/2564676547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e0107270 0x7ff2e010ee40 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8009ca0 tx=0x7ff2d802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/2564676547 shutdown_connections 2024-04-25T21:00:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/2564676547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e0107270 0x7ff2e010ee40 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/2564676547 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e0102b50 0x7ff2e0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/2564676547 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e01021a0 0x7ff2e0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/2564676547 >> 172.21.15.96:0/2564676547 conn(0x7ff2e00fda00 msgr2=0x7ff2e00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/2564676547 shutdown_connections 2024-04-25T21:00:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.267+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/2564676547 wait complete. 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 Processor -- start 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 -- start start 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 0x7ff2e01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e0107270 0x7ff2e0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e019cf50 0x7ff2e019d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01121c0 con 0x7ff2e01021a0 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0112040 con 0x7ff2e019cf50 2024-04-25T21:00:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e7fcd640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e0112340 con 0x7ff2e0107270 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e5d69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 0x7ff2e01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.268+0000 7ff2e5d69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 0x7ff2e01986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54180/0 (socket says 172.21.15.96:54180) 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 -- 172.21.15.96:0/714560213 learned_addr learned my addr 172.21.15.96:0/714560213 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e656a640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e019cf50 0x7ff2e019d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5568640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e0107270 0x7ff2e0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 -- 172.21.15.96:0/714560213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e0107270 msgr2=0x7ff2e0198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e0107270 0x7ff2e0198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 -- 172.21.15.96:0/714560213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e019cf50 msgr2=0x7ff2e019d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e019cf50 0x7ff2e019d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 -- 172.21.15.96:0/714560213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2e019da40 con 0x7ff2e01021a0 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e656a640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e019cf50 0x7ff2e019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 0x7ff2e01986e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d950 tx=0x7ff2dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:25.272 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2d6ffd640 1 -- 172.21.15.96:0/714560213 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc017070 con 0x7ff2e01021a0 2024-04-25T21:00:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2e019dd30 con 0x7ff2e01021a0 2024-04-25T21:00:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2d6ffd640 1 -- 172.21.15.96:0/714560213 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc009c30 con 0x7ff2e01021a0 2024-04-25T21:00:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2d6ffd640 1 -- 172.21.15.96:0/714560213 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc00f620 con 0x7ff2e01021a0 2024-04-25T21:00:25.273 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.269+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2e01a5520 con 0x7ff2e01021a0 2024-04-25T21:00:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.272+0000 7ff2d6ffd640 1 -- 172.21.15.96:0/714560213 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff2dc010460 con 0x7ff2e01021a0 2024-04-25T21:00:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.272+0000 7ff2eadc2740 1 Processor -- start 2024-04-25T21:00:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 -- start start 2024-04-25T21:00:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9620 0x55cf5cea9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf5ce0e970 con 0x55cf5ceb4800 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cf5cdab4a0 con 0x55cf5cea9620 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf5ce0a930 con 0x55cf5cea9f40 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9620 0x55cf5cea9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54182/0 (socket says 172.21.15.96:54182) 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 -- 172.21.15.96:0/732736986 learned_addr learned my addr 172.21.15.96:0/732736986 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9620 0x55cf5cea9a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:44320/0 (socket says 172.21.15.96:44320) 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2d6ffd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff2cc0a3dc0 0x7ff2cc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e656a640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff2cc0a3dc0 0x7ff2cc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 -- 172.21.15.96:0/732736986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 msgr2=0x55cf5ceb42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2d6ffd640 1 -- 172.21.15.96:0/714560213 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7ff2dc0d0320 con 0x7ff2e01021a0 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 -- 172.21.15.96:0/732736986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9620 msgr2=0x55cf5cea9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9620 0x55cf5cea9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 -- 172.21.15.96:0/732736986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5cb21210 con 0x55cf5ceb4800 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.273+0000 7ff2e5568640 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb6bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009c20 tx=0x7ff2d0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc78f1b99fd78a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2cbfff640 1 -- 172.21.15.96:0/732736986 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004050 con 0x55cf5ceb4800 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2e656a640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff2cc0a3dc0 0x7ff2cc0a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006010 tx=0x7ff2d802a8c0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2cbfff640 1 -- 172.21.15.96:0/732736986 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00054d0 con 0x55cf5ceb4800 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2cbfff640 1 -- 172.21.15.96:0/732736986 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d0004d00 con 0x55cf5ceb4800 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2eadc2740 1 -- 172.21.15.96:0/732736986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 msgr2=0x55cf5ceb6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb6bf0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009c20 tx=0x7ff2d0011810 comp rx=0 tx=0).stop 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2eadc2740 1 -- 172.21.15.96:0/732736986 shutdown_connections 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb6bf0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.274+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/732736986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9620 0x55cf5cea9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- 172.21.15.96:0/732736986 >> 172.21.15.96:0/732736986 conn(0x55cf5cea5b30 msgr2=0x55cf5cea5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- 172.21.15.96:0/732736986 shutdown_connections 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- 172.21.15.96:0/732736986 wait complete. 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- start start 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 Processor -- start 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5ceb4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf5ceb7430 con 0x55cf5ceb4800 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55cf5cda90b0 con 0x55cf5cea9f40 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55cf5ceb75b0 con 0x55cf5cea9620 2024-04-25T21:00:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.275+0000 7ff2eadc2740 10 client.0 _ll_register_callbacks cb 0x55cf5cd96d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5ceb4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5568640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54196/0 (socket says 172.21.15.96:54196) 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e656a640 1 -- 172.21.15.96:0/3683803852 learned_addr learned my addr 172.21.15.96:0/3683803852 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 -- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 msgr2=0x55cf5ceb1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 -- 172.21.15.96:0/3683803852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 msgr2=0x55cf5ceb19c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb19c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 -- 172.21.15.96:0/3683803852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5cbed620 con 0x55cf5cea9620 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e656a640 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5ceb4170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0cdae0 tx=0x7ff2dc09bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf412f0da43350 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T21:00:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3683803852 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020070 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2d00054d0 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3683803852 <== mon.1 v2:172.21.15.139:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7ff2dc098a50 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3683803852 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2dc09c3e0 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.276+0000 7ff2eadc2740 10 client.25081 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf5cec3410 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3683803852 <== mon.1 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d00054d0 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3683803852 <== mon.1 v2:172.21.15.139:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff2dc0a4b10 con 0x55cf5cea9620 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 10 client.25081 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 10 client.25081 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 20 client.25081 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cec48d0 0x55cf5cec6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 4 client.25081 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf5ce1c390 con 0x55cf5cec48d0 2024-04-25T21:00:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.277+0000 7ff2e5568640 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cec48d0 0x55cf5cec6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.278+0000 7ff2c97fa640 10 client.25081 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.278+0000 7ff2e5568640 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cec48d0 0x55cf5cec6ce0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1206c3b9ac1cd443 server_cookie=9d47bc1e2c026931 in_seq=0 out_seq=0 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.282+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3683803852 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cf5ce1c390 con 0x55cf5cec48d0 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.282+0000 7ff2c97fa640 10 client.25081 handle_command_reply: tid=0 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 client.25081 shutdown 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 msgr2=0x55cf5ceb4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5ceb4170 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc0cdae0 tx=0x7ff2dc09bf20 comp rx=0 tx=0).stop 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 shutdown_connections 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cec48d0 0x55cf5cec6ce0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceb19c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2c97fa640 0 client.25081 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3683803852 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5ceb4170 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 >> 172.21.15.96:0/3683803852 conn(0x55cf5cea5b30 msgr2=0x55cf5ceab860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 shutdown_connections 2024-04-25T21:00:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3683803852 wait complete. 2024-04-25T21:00:25.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.283+0000 7ff2eadc2740 20 client.25081 trim_cache size 0 max 16384 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 Processor -- start 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 -- start start 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 0x55cf5cec02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5cecc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5ceb4800 0x55cf5ceccb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01b0270 con 0x55cf5cea9620 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01adb40 con 0x55cf5ceb4800 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01ab9b0 con 0x55cf5cea9f40 2024-04-25T21:00:25.304 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2e5d69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 0x55cf5cec02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2e5d69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 0x55cf5cec02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54212/0 (socket says 172.21.15.96:54212) 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2e5d69640 1 -- 172.21.15.96:0/391069559 learned_addr learned my addr 172.21.15.96:0/391069559 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5ceb4800 0x55cf5ceccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.301+0000 7ff2e5568640 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5cecc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e5d69640 1 -- 172.21.15.96:0/391069559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 msgr2=0x55cf5cecc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5cecc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e5d69640 1 -- 172.21.15.96:0/391069559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5ceb4800 msgr2=0x55cf5ceccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5ceb4800 0x55cf5ceccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e5d69640 1 -- 172.21.15.96:0/391069559 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf5cea9d70 con 0x55cf5cea9620 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e656a640 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5ceb4800 0x55cf5ceccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 0x55cf5cec02f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09c370 tx=0x7ff2dc09a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed4e51cbac92732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2cb7fe640 1 -- 172.21.15.96:0/391069559 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc020050 con 0x55cf5cea9620 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2cb7fe640 1 -- 172.21.15.96:0/391069559 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc0b5020 con 0x55cf5cea9620 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.302+0000 7ff2cb7fe640 1 -- 172.21.15.96:0/391069559 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2dc098320 con 0x55cf5cea9620 2024-04-25T21:00:25.305 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 -- 172.21.15.96:0/391069559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 msgr2=0x55cf5cec02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 0x55cf5cec02f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc09c370 tx=0x7ff2dc09a8f0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 -- 172.21.15.96:0/391069559 shutdown_connections 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5ceb4800 0x55cf5ceccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9f40 0x55cf5cecc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/391069559 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5cea9620 0x55cf5cec02f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 -- 172.21.15.96:0/391069559 >> 172.21.15.96:0/391069559 conn(0x55cf5cea5b30 msgr2=0x55cf5cea4420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 -- 172.21.15.96:0/391069559 shutdown_connections 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 -- 172.21.15.96:0/391069559 wait complete. 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.303+0000 7ff2eadc2740 1 -- start start 2024-04-25T21:00:25.306 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 Processor -- start 2024-04-25T21:00:25.308 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5cebcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceba3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01ac000 con 0x55cf5ceb4800 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01ad470 con 0x55cf5cea9f40 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7ff2e01ac6d0 con 0x55cf5cea9620 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2eadc2740 10 client.0 _ll_register_callbacks cb 0x7ff2dc09c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceba3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2e5568640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2e5d69640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5cebcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2e5568640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:44338/0 (socket says 172.21.15.96:44338) 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2e656a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceba3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54228/0 (socket says 172.21.15.96:54228) 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.304+0000 7ff2e5568640 1 -- 172.21.15.96:0/3473885325 learned_addr learned my addr 172.21.15.96:0/3473885325 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2e656a640 1 -- 172.21.15.96:0/3473885325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 msgr2=0x55cf5cebcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2e656a640 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5cebcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2e656a640 1 -- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 msgr2=0x55cf5ceb9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2e656a640 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2e656a640 1 -- 172.21.15.96:0/3473885325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf5cebaa90 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2e656a640 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceba3a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2d802ade0 tx=0x7ff2d80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a73f78bfd7db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3473885325 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d8003e20 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3473885325 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8411+0+0 (secure 0 0 0) 0x7ff2d802f870 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2dc0b5020 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3473885325 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff2d8004810 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2eadc2740 10 client.15651 fetch_fsmap learned FSMap version 27 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.305+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55cf5cf2f190 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3473885325 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2dc0b5020 con 0x55cf5ceb4800 2024-04-25T21:00:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3473885325 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8346+0+0 (secure 0 0 0) 0x7ff2d804b040 con 0x55cf5ceb4800 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2eadc2740 10 client.15651 fetch_fsmap finished waiting for FSMap version 27 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2eadc2740 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2eadc2740 20 client.15651 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cf2f970 0x55cf5cf31d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2eadc2740 4 client.15651 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf5ce16200 con 0x55cf5cf2f970 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cf2f970 0x55cf5cf31d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2e5d69640 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cf2f970 0x55cf5cf31d90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6768c3a8006938e1 server_cookie=375acedcf81b9ef6 in_seq=0 out_seq=0 2024-04-25T21:00:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.306+0000 7ff2c97fa640 10 client.15651 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.711+0000 7ff2c97fa640 1 -- 172.21.15.96:0/3473885325 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cf5ce16200 con 0x55cf5cf2f970 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.711+0000 7ff2c97fa640 10 client.15651 handle_command_reply: tid=0 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.711+0000 7ff2eadc2740 1 client.15651 shutdown 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.711+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 msgr2=0x55cf5ceba3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.711+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceba3a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7ff2d802ade0 tx=0x7ff2d80095c0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 shutdown_connections 2024-04-25T21:00:25.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55cf5cf2f970 0x55cf5cf31d90 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf5ceb4800 0x55cf5ceba3a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55cf5cea9f40 0x55cf5ceb9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2c97fa640 0 client.15651 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 --2- 172.21.15.96:0/3473885325 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55cf5cea9620 0x55cf5cebcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 >> 172.21.15.96:0/3473885325 conn(0x55cf5cea5b30 msgr2=0x55cf5cea4cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 shutdown_connections 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 1 -- 172.21.15.96:0/3473885325 wait complete. 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2eadc2740 20 client.15651 trim_cache size 0 max 16384 2024-04-25T21:00:25.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff2cc0a3dc0 msgr2=0x7ff2cc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.715 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff2cc0a3dc0 0x7ff2cc0a6280 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff2d8006010 tx=0x7ff2d802a8c0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 msgr2=0x7ff2e01986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.712+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 0x7ff2e01986e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc00d950 tx=0x7ff2dc00de10 comp rx=0 tx=0).stop 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 shutdown_connections 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7ff2cc0a3dc0 0x7ff2cc0a6280 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff2e019cf50 0x7ff2e019d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7ff2e0107270 0x7ff2e0198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 --2- 172.21.15.96:0/714560213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2e01021a0 0x7ff2e01986e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 >> 172.21.15.96:0/714560213 conn(0x7ff2e00fda00 msgr2=0x7ff2e00ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 shutdown_connections 2024-04-25T21:00:25.716 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:25.713+0000 7ff2e7fcd640 1 -- 172.21.15.96:0/714560213 wait complete. 2024-04-25T21:00:25.738 INFO:tasks.workunit.client.0.smithi096.stdout:checking build system type... x86_64-pc-linux-gnu 2024-04-25T21:00:25.738 INFO:tasks.workunit.client.0.smithi096.stdout:checking host system type... x86_64-pc-linux-gnu 2024-04-25T21:00:25.739 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to print strings... printf 2024-04-25T21:00:25.739 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc... gcc 2024-04-25T21:00:25.826 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether the C compiler works... yes 2024-04-25T21:00:25.826 INFO:tasks.workunit.client.0.smithi096.stdout:checking for C compiler default output file name... a.out 2024-04-25T21:00:25.864 INFO:tasks.workunit.client.0.smithi096.stdout:checking for suffix of executables... 2024-04-25T21:00:25.900 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are cross compiling... no 2024-04-25T21:00:25.941 INFO:tasks.workunit.client.0.smithi096.stdout:checking for suffix of object files... o 2024-04-25T21:00:25.965 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are using the GNU C compiler... yes 2024-04-25T21:00:25.988 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc accepts -g... yes 2024-04-25T21:00:26.071 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to accept ISO C89... none needed 2024-04-25T21:00:26.075 INFO:tasks.workunit.client.0.smithi096.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-04-25T21:00:26.079 INFO:tasks.workunit.client.0.smithi096.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-04-25T21:00:26.080 INFO:tasks.workunit.client.0.smithi096.stdout:checking for egrep... /usr/bin/grep -E 2024-04-25T21:00:26.081 INFO:tasks.workunit.client.0.smithi096.stdout:checking for fgrep... /usr/bin/grep -F 2024-04-25T21:00:26.084 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ld used by gcc... /usr/bin/ld 2024-04-25T21:00:26.086 INFO:tasks.workunit.client.0.smithi096.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-04-25T21:00:26.101 INFO:tasks.workunit.client.0.smithi096.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-04-25T21:00:26.316 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:00:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: ignoring --setuser ceph since I am not root 2024-04-25T21:00:26.316 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:00:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: ignoring --setgroup ceph since I am not root 2024-04-25T21:00:26.316 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:00:26 smithi139 ceph-mds[63297]: -- 172.21.15.139:0/2098443595 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562dfb227a40 con 0x562dfb1fb000 2024-04-25T21:00:26.316 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:00:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: starting mds.c at 2024-04-25T21:00:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:26 smithi097 ceph-mon[32667]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-25T21:00:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:26 smithi097 ceph-mon[32667]: daemon mds.c restarted 2024-04-25T21:00:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:26 smithi139 ceph-mon[33027]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-25T21:00:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:26 smithi139 ceph-mon[33027]: daemon mds.c restarted 2024-04-25T21:00:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:26 smithi096 ceph-mon[29288]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.2 MiB/s wr, 394 op/s 2024-04-25T21:00:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:26 smithi096 ceph-mon[29288]: daemon mds.c restarted 2024-04-25T21:00:27.000 INFO:tasks.workunit.client.0.smithi096.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-04-25T21:00:27.000 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether ln -s works... yes 2024-04-25T21:00:27.012 INFO:tasks.workunit.client.0.smithi096.stdout:checking the maximum length of command line arguments... 1572864 2024-04-25T21:00:27.015 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-04-25T21:00:27.015 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-04-25T21:00:27.015 INFO:tasks.workunit.client.0.smithi096.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-04-25T21:00:27.016 INFO:tasks.workunit.client.0.smithi096.stdout:checking for objdump... objdump 2024-04-25T21:00:27.017 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to recognize dependent libraries... pass_all 2024-04-25T21:00:27.017 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dlltool... no 2024-04-25T21:00:27.017 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-04-25T21:00:27.017 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ar... ar 2024-04-25T21:00:27.116 INFO:tasks.workunit.client.0.smithi096.stdout:checking for archiver @FILE support... @ 2024-04-25T21:00:27.117 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strip... strip 2024-04-25T21:00:27.117 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ranlib... ranlib 2024-04-25T21:00:27.117 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gawk... gawk 2024-04-25T21:00:27.216 INFO:tasks.workunit.client.0.smithi096.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-04-25T21:00:27.217 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sysroot... no 2024-04-25T21:00:27.229 INFO:tasks.workunit.client.0.smithi096.stdout:checking for a working dd... /usr/bin/dd 2024-04-25T21:00:27.240 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-04-25T21:00:27.274 INFO:tasks.workunit.client.0.smithi096.stdout:checking for mt... no 2024-04-25T21:00:27.286 INFO:tasks.workunit.client.0.smithi096.stdout:checking if : is a manifest tool... no 2024-04-25T21:00:27.387 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to run the C preprocessor... gcc -E 2024-04-25T21:00:27.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:27 smithi097 ceph-mon[32667]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:00:27.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:27 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6836/2108619178,v1:172.21.15.139:6837/2108619178] up:boot 2024-04-25T21:00:27.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:27 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T21:00:27.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:27 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:00:27.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:27 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:00:27.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:27 smithi139 ceph-mon[33027]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:00:27.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:27 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6836/2108619178,v1:172.21.15.139:6837/2108619178] up:boot 2024-04-25T21:00:27.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:27 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T21:00:27.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:27 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:00:27.623 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:27 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:00:27.685 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ANSI C header files... yes 2024-04-25T21:00:27.723 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/types.h... yes 2024-04-25T21:00:27.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:27 smithi096 ceph-mon[29288]: osdmap e95: 12 total, 12 up, 12 in 2024-04-25T21:00:27.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:27 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6836/2108619178,v1:172.21.15.139:6837/2108619178] up:boot 2024-04-25T21:00:27.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:27 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 2 up:standby-replay 7 up:standby 2024-04-25T21:00:27.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:27 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:00:27.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:27 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:00:27.763 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/stat.h... yes 2024-04-25T21:00:27.804 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdlib.h... yes 2024-04-25T21:00:27.846 INFO:tasks.workunit.client.0.smithi096.stdout:checking for string.h... yes 2024-04-25T21:00:27.891 INFO:tasks.workunit.client.0.smithi096.stdout:checking for memory.h... yes 2024-04-25T21:00:27.935 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strings.h... yes 2024-04-25T21:00:27.979 INFO:tasks.workunit.client.0.smithi096.stdout:checking for inttypes.h... yes 2024-04-25T21:00:28.024 INFO:tasks.workunit.client.0.smithi096.stdout:checking for stdint.h... yes 2024-04-25T21:00:28.070 INFO:tasks.workunit.client.0.smithi096.stdout:checking for unistd.h... yes 2024-04-25T21:00:28.121 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dlfcn.h... yes 2024-04-25T21:00:28.128 INFO:tasks.workunit.client.0.smithi096.stdout:checking for objdir... .libs 2024-04-25T21:00:28.254 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-04-25T21:00:28.255 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-04-25T21:00:28.299 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-04-25T21:00:28.346 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc static flag -static works... no 2024-04-25T21:00:28.411 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc supports -c -o file.o... yes 2024-04-25T21:00:28.411 INFO:tasks.workunit.client.0.smithi096.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-04-25T21:00:28.417 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-04-25T21:00:28.468 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether -lc should be explicitly linked in... no 2024-04-25T21:00:28.527 INFO:tasks.workunit.client.0.smithi096.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-04-25T21:00:28.528 INFO:tasks.workunit.client.0.smithi096.stdout:checking how to hardcode library paths into programs... immediate 2024-04-25T21:00:28.529 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether stripping libraries is possible... yes 2024-04-25T21:00:28.529 INFO:tasks.workunit.client.0.smithi096.stdout:checking if libtool supports shared libraries... yes 2024-04-25T21:00:28.529 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether to build shared libraries... yes 2024-04-25T21:00:28.529 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether to build static libraries... yes 2024-04-25T21:00:28.535 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc... (cached) gcc 2024-04-25T21:00:28.571 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-04-25T21:00:28.571 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc accepts -g... (cached) yes 2024-04-25T21:00:28.571 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-04-25T21:00:28.571 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gmake... /usr/bin/gmake 2024-04-25T21:00:28.571 INFO:tasks.workunit.client.0.smithi096.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-04-25T21:00:28.572 INFO:tasks.workunit.client.0.smithi096.stdout:checking for tar... /bin/tar 2024-04-25T21:00:28.572 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gzip... /bin/gzip 2024-04-25T21:00:28.598 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether gcc -MM is supported... yes 2024-04-25T21:00:28.598 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sort... /bin/sort 2024-04-25T21:00:28.598 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether ln -s works... yes 2024-04-25T21:00:28.599 INFO:tasks.workunit.client.0.smithi096.stdout:checking for rpm... /bin/rpm 2024-04-25T21:00:28.601 INFO:tasks.workunit.client.0.smithi096.stdout:checking for rpmbuild... no 2024-04-25T21:00:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:28 smithi097 ceph-mon[32667]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 433 op/s 2024-04-25T21:00:28.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:28 smithi139 ceph-mon[33027]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 433 op/s 2024-04-25T21:00:28.659 INFO:tasks.workunit.client.0.smithi096.stdout:checking assert.h usability... yes 2024-04-25T21:00:28.675 INFO:tasks.workunit.client.0.smithi096.stdout:checking assert.h presence... yes 2024-04-25T21:00:28.675 INFO:tasks.workunit.client.0.smithi096.stdout:checking for assert.h... yes 2024-04-25T21:00:28.727 INFO:tasks.workunit.client.0.smithi096.stdout:checking bstring.h usability... no 2024-04-25T21:00:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:28 smithi096 ceph-mon[29288]: pgmap v88: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 433 op/s 2024-04-25T21:00:28.756 INFO:tasks.workunit.client.0.smithi096.stdout:checking bstring.h presence... no 2024-04-25T21:00:28.756 INFO:tasks.workunit.client.0.smithi096.stdout:checking for bstring.h... no 2024-04-25T21:00:28.796 INFO:tasks.workunit.client.0.smithi096.stdout:checking libgen.h usability... yes 2024-04-25T21:00:28.812 INFO:tasks.workunit.client.0.smithi096.stdout:checking libgen.h presence... yes 2024-04-25T21:00:28.812 INFO:tasks.workunit.client.0.smithi096.stdout:checking for libgen.h... yes 2024-04-25T21:00:28.870 INFO:tasks.workunit.client.0.smithi096.stdout:checking dirent.h usability... yes 2024-04-25T21:00:28.887 INFO:tasks.workunit.client.0.smithi096.stdout:checking dirent.h presence... yes 2024-04-25T21:00:28.888 INFO:tasks.workunit.client.0.smithi096.stdout:checking for dirent.h... yes 2024-04-25T21:00:28.933 INFO:tasks.workunit.client.0.smithi096.stdout:checking errno.h usability... yes 2024-04-25T21:00:28.950 INFO:tasks.workunit.client.0.smithi096.stdout:checking errno.h presence... yes 2024-04-25T21:00:28.950 INFO:tasks.workunit.client.0.smithi096.stdout:checking for errno.h... yes 2024-04-25T21:00:28.996 INFO:tasks.workunit.client.0.smithi096.stdout:checking malloc.h usability... yes 2024-04-25T21:00:29.014 INFO:tasks.workunit.client.0.smithi096.stdout:checking malloc.h presence... yes 2024-04-25T21:00:29.014 INFO:tasks.workunit.client.0.smithi096.stdout:checking for malloc.h... yes 2024-04-25T21:00:29.067 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid.h usability... no 2024-04-25T21:00:29.097 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid.h presence... no 2024-04-25T21:00:29.097 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid.h... no 2024-04-25T21:00:29.147 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid/uuid.h usability... yes 2024-04-25T21:00:29.166 INFO:tasks.workunit.client.0.smithi096.stdout:checking uuid/uuid.h presence... yes 2024-04-25T21:00:29.166 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid/uuid.h... yes 2024-04-25T21:00:29.218 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/uuid.h usability... no 2024-04-25T21:00:29.248 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/uuid.h presence... no 2024-04-25T21:00:29.248 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/uuid.h... no 2024-04-25T21:00:29.296 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/file.h usability... yes 2024-04-25T21:00:29.313 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/file.h presence... yes 2024-04-25T21:00:29.313 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/file.h... yes 2024-04-25T21:00:29.378 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fcntl.h usability... yes 2024-04-25T21:00:29.395 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fcntl.h presence... yes 2024-04-25T21:00:29.395 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fcntl.h... yes 2024-04-25T21:00:29.450 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/syssgi.h usability... no 2024-04-25T21:00:29.478 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/syssgi.h presence... no 2024-04-25T21:00:29.478 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/syssgi.h... no 2024-04-25T21:00:29.539 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/param.h usability... yes 2024-04-25T21:00:29.558 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/param.h presence... yes 2024-04-25T21:00:29.558 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/param.h... yes 2024-04-25T21:00:29.573 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/stat.h... (cached) yes 2024-04-25T21:00:29.609 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/statvfs.h usability... yes 2024-04-25T21:00:29.626 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/statvfs.h presence... yes 2024-04-25T21:00:29.626 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/statvfs.h... yes 2024-04-25T21:00:29.673 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/time.h usability... yes 2024-04-25T21:00:29.690 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/time.h presence... yes 2024-04-25T21:00:29.690 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/time.h... yes 2024-04-25T21:00:29.741 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/ioctl.h usability... yes 2024-04-25T21:00:29.758 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/ioctl.h presence... yes 2024-04-25T21:00:29.758 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/ioctl.h... yes 2024-04-25T21:00:29.812 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/wait.h usability... yes 2024-04-25T21:00:29.830 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/wait.h presence... yes 2024-04-25T21:00:29.830 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/wait.h... yes 2024-04-25T21:00:29.845 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/types.h... (cached) yes 2024-04-25T21:00:29.849 INFO:tasks.workunit.client.0.smithi096.stdout:checking for strings.h... (cached) yes 2024-04-25T21:00:29.886 INFO:tasks.workunit.client.0.smithi096.stdout:checking err.h usability... yes 2024-04-25T21:00:29.903 INFO:tasks.workunit.client.0.smithi096.stdout:checking err.h presence... yes 2024-04-25T21:00:29.903 INFO:tasks.workunit.client.0.smithi096.stdout:checking for err.h... yes 2024-04-25T21:00:29.950 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/falloc.h usability... yes 2024-04-25T21:00:29.965 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/falloc.h presence... yes 2024-04-25T21:00:29.966 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/falloc.h... yes 2024-04-25T21:00:30.018 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-04-25T21:00:30.048 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-04-25T21:00:30.048 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fs/xfs_fsops.h... no 2024-04-25T21:00:30.093 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_itable.h usability... no 2024-04-25T21:00:30.122 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/fs/xfs_itable.h presence... no 2024-04-25T21:00:30.122 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/fs/xfs_itable.h... no 2024-04-25T21:00:30.167 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/platform_defs.h usability... no 2024-04-25T21:00:30.196 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/platform_defs.h presence... no 2024-04-25T21:00:30.196 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/platform_defs.h... no 2024-04-25T21:00:30.241 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfs/ioctl.h usability... no 2024-04-25T21:00:30.270 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfs/ioctl.h presence... no 2024-04-25T21:00:30.270 INFO:tasks.workunit.client.0.smithi096.stdout:checking for btrfs/ioctl.h... no 2024-04-25T21:00:30.314 INFO:tasks.workunit.client.0.smithi096.stdout:checking cifs/ioctl.h usability... no 2024-04-25T21:00:30.343 INFO:tasks.workunit.client.0.smithi096.stdout:checking cifs/ioctl.h presence... no 2024-04-25T21:00:30.343 INFO:tasks.workunit.client.0.smithi096.stdout:checking for cifs/ioctl.h... no 2024-04-25T21:00:30.389 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/mman.h usability... yes 2024-04-25T21:00:30.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/mman.h presence... yes 2024-04-25T21:00:30.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/mman.h... yes 2024-04-25T21:00:30.457 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/ext4.h usability... no 2024-04-25T21:00:30.484 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/ext4.h presence... no 2024-04-25T21:00:30.484 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/ext4.h... no 2024-04-25T21:00:30.512 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/xfs_log_format.h... no 2024-04-25T21:00:30.565 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid_compare... no 2024-04-25T21:00:30.604 INFO:tasks.workunit.client.0.smithi096.stdout:checking for uuid_compare in -luuid... yes 2024-04-25T21:00:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:30 smithi097 ceph-mon[32667]: pgmap v89: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 433 op/s 2024-04-25T21:00:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:30 smithi139 ceph-mon[33027]: pgmap v89: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 433 op/s 2024-04-25T21:00:30.693 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/xfs.h... yes 2024-04-25T21:00:30.723 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/libxfs.h... no 2024-04-25T21:00:30.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:30 smithi096 ceph-mon[29288]: pgmap v89: 97 pgs: 97 active+clean; 42 MiB data, 567 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.2 MiB/s wr, 433 op/s 2024-04-25T21:00:30.753 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether xlog_assign_lsn is declared... no 2024-04-25T21:00:30.791 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/xqm.h... yes 2024-04-25T21:00:30.842 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfsctl from xfs/xfs.h... ok 2024-04-25T21:00:30.879 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/handle.h usability... yes 2024-04-25T21:00:30.894 INFO:tasks.workunit.client.0.smithi096.stdout:checking xfs/handle.h presence... yes 2024-04-25T21:00:30.894 INFO:tasks.workunit.client.0.smithi096.stdout:checking for xfs/handle.h... yes 2024-04-25T21:00:30.934 INFO:tasks.workunit.client.0.smithi096.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-04-25T21:00:30.977 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/acl.h usability... yes 2024-04-25T21:00:30.994 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/acl.h presence... yes 2024-04-25T21:00:30.995 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/acl.h... yes 2024-04-25T21:00:31.036 INFO:tasks.workunit.client.0.smithi096.stdout:checking acl/libacl.h usability... yes 2024-04-25T21:00:31.054 INFO:tasks.workunit.client.0.smithi096.stdout:checking acl/libacl.h presence... yes 2024-04-25T21:00:31.055 INFO:tasks.workunit.client.0.smithi096.stdout:checking for acl/libacl.h... yes 2024-04-25T21:00:31.095 INFO:tasks.workunit.client.0.smithi096.stdout:checking for acl_init in -lacl... yes 2024-04-25T21:00:31.170 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm-ndbm.h usability... no 2024-04-25T21:00:31.200 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm-ndbm.h presence... no 2024-04-25T21:00:31.200 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gdbm-ndbm.h... no 2024-04-25T21:00:31.241 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm/ndbm.h usability... no 2024-04-25T21:00:31.275 INFO:tasks.workunit.client.0.smithi096.stdout:checking gdbm/ndbm.h presence... no 2024-04-25T21:00:31.275 INFO:tasks.workunit.client.0.smithi096.stdout:checking for gdbm/ndbm.h... no 2024-04-25T21:00:31.317 INFO:tasks.workunit.client.0.smithi096.stdout:checking ndbm.h usability... no 2024-04-25T21:00:31.352 INFO:tasks.workunit.client.0.smithi096.stdout:checking ndbm.h presence... no 2024-04-25T21:00:31.352 INFO:tasks.workunit.client.0.smithi096.stdout:checking for ndbm.h... no 2024-04-25T21:00:31.388 INFO:tasks.workunit.client.0.smithi096.stdout:checking libaio.h usability... yes 2024-04-25T21:00:31.411 INFO:tasks.workunit.client.0.smithi096.stdout:checking libaio.h presence... yes 2024-04-25T21:00:31.411 INFO:tasks.workunit.client.0.smithi096.stdout:checking for libaio.h... yes 2024-04-25T21:00:31.459 INFO:tasks.workunit.client.0.smithi096.stdout:checking liburing.h usability... no 2024-04-25T21:00:31.492 INFO:tasks.workunit.client.0.smithi096.stdout:checking liburing.h presence... no 2024-04-25T21:00:31.493 INFO:tasks.workunit.client.0.smithi096.stdout:checking for liburing.h... no 2024-04-25T21:00:31.534 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/capability.h usability... no 2024-04-25T21:00:31.566 INFO:tasks.workunit.client.0.smithi096.stdout:checking sys/capability.h presence... no 2024-04-25T21:00:31.567 INFO:tasks.workunit.client.0.smithi096.stdout:checking for sys/capability.h... no 2024-04-25T21:00:31.598 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fiemap.h usability... yes 2024-04-25T21:00:31.614 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fiemap.h presence... yes 2024-04-25T21:00:31.614 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/fiemap.h... yes 2024-04-25T21:00:31.651 INFO:tasks.workunit.client.0.smithi096.stdout:checking for fallocate... yes 2024-04-25T21:00:31.696 INFO:tasks.workunit.client.0.smithi096.stdout:checking for open_by_handle_at... yes 2024-04-25T21:00:31.733 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fs.h usability... yes 2024-04-25T21:00:31.749 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/fs.h presence... yes 2024-04-25T21:00:31.749 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/fs.h... yes 2024-04-25T21:00:31.807 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfsutil.h usability... no 2024-04-25T21:00:31.835 INFO:tasks.workunit.client.0.smithi096.stdout:checking btrfsutil.h presence... no 2024-04-25T21:00:31.835 INFO:tasks.workunit.client.0.smithi096.stdout:checking for btrfsutil.h... no 2024-04-25T21:00:31.882 INFO:tasks.workunit.client.0.smithi096.stdout:checking for copy_file_range... yes 2024-04-25T21:00:31.926 INFO:tasks.workunit.client.0.smithi096.stdout:checking for SEEK_DATA... yes 2024-04-25T21:00:31.984 INFO:tasks.workunit.client.0.smithi096.stdout:checking for BMV_OF_SHARED... yes 2024-04-25T21:00:32.029 INFO:tasks.workunit.client.0.smithi096.stdout:checking for nftw... yes 2024-04-25T21:00:32.083 INFO:tasks.workunit.client.0.smithi096.stdout:checking for RLIMIT_NOFILE... yes 2024-04-25T21:00:32.126 INFO:tasks.workunit.client.0.smithi096.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-04-25T21:00:32.188 INFO:tasks.workunit.client.0.smithi096.stdout:checking for FICLONE... yes 2024-04-25T21:00:32.234 INFO:tasks.workunit.client.0.smithi096.stdout:checking for renameat2... yes 2024-04-25T21:00:32.275 INFO:tasks.workunit.client.0.smithi096.stdout:checking for reallocarray... yes 2024-04-25T21:00:32.340 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct mount_attr... yes 2024-04-25T21:00:32.406 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_qgroup_limit... yes 2024-04-25T21:00:32.462 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-04-25T21:00:32.520 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-04-25T21:00:32.576 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-04-25T21:00:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:32 smithi097 ceph-mon[32667]: pgmap v90: 97 pgs: 97 active+clean; 30 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 967 op/s 2024-04-25T21:00:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:32 smithi139 ceph-mon[33027]: pgmap v90: 97 pgs: 97 active+clean; 30 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 967 op/s 2024-04-25T21:00:32.713 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-04-25T21:00:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:32 smithi096 ceph-mon[29288]: pgmap v90: 97 pgs: 97 active+clean; 30 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 967 op/s 2024-04-25T21:00:32.838 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-04-25T21:00:32.894 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-04-25T21:00:32.933 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs.h usability... yes 2024-04-25T21:00:32.949 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs.h presence... yes 2024-04-25T21:00:32.950 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/btrfs.h... yes 2024-04-25T21:00:32.997 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs_tree.h usability... yes 2024-04-25T21:00:33.014 INFO:tasks.workunit.client.0.smithi096.stdout:checking linux/btrfs_tree.h presence... yes 2024-04-25T21:00:33.015 INFO:tasks.workunit.client.0.smithi096.stdout:checking for linux/btrfs_tree.h... yes 2024-04-25T21:00:33.054 INFO:tasks.workunit.client.0.smithi096.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-04-25T21:00:33.084 INFO:tasks.workunit.client.0.smithi096.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-04-25T21:00:33.103 INFO:tasks.workunit.client.0.smithi096.stdout:configure: creating ./config.status 2024-04-25T21:00:33.418 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating include/builddefs 2024-04-25T21:00:33.431 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: creating include/config.h 2024-04-25T21:00:33.442 INFO:tasks.workunit.client.0.smithi096.stdout:config.status: executing libtool commands 2024-04-25T21:00:33.523 INFO:tasks.workunit.client.0.smithi096.stdout:Building include 2024-04-25T21:00:33.523 INFO:tasks.workunit.client.0.smithi096.stdout:Building lib 2024-04-25T21:00:33.524 INFO:tasks.workunit.client.0.smithi096.stdout:Building tests 2024-04-25T21:00:33.532 INFO:tasks.workunit.client.0.smithi096.stdout: [LTDEP] 2024-04-25T21:00:33.533 INFO:tasks.workunit.client.0.smithi096.stdout:Building btrfs 2024-04-25T21:00:33.534 INFO:tasks.workunit.client.0.smithi096.stdout:Building ceph 2024-04-25T21:00:33.534 INFO:tasks.workunit.client.0.smithi096.stdout:Building cifs 2024-04-25T21:00:33.544 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-04-25T21:00:33.544 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-04-25T21:00:33.563 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-04-25T21:00:33.563 INFO:tasks.workunit.client.0.smithi096.stdout:Building ext4 2024-04-25T21:00:33.565 INFO:tasks.workunit.client.0.smithi096.stdout:Building f2fs 2024-04-25T21:00:33.574 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-04-25T21:00:33.588 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-04-25T21:00:33.599 INFO:tasks.workunit.client.0.smithi096.stdout:Building generic 2024-04-25T21:00:33.637 INFO:tasks.workunit.client.0.smithi096.stdout:Building nfs 2024-04-25T21:00:33.644 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-04-25T21:00:33.660 INFO:tasks.workunit.client.0.smithi096.stdout:Building ocfs2 2024-04-25T21:00:33.668 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-04-25T21:00:33.683 INFO:tasks.workunit.client.0.smithi096.stdout:Building overlay 2024-04-25T21:00:33.712 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-04-25T21:00:33.719 INFO:tasks.workunit.client.0.smithi096.stdout:Building perf 2024-04-25T21:00:33.725 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-04-25T21:00:33.739 INFO:tasks.workunit.client.0.smithi096.stdout:Building selftest 2024-04-25T21:00:33.740 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-04-25T21:00:33.745 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-04-25T21:00:33.755 INFO:tasks.workunit.client.0.smithi096.stdout:Building shared 2024-04-25T21:00:33.761 INFO:tasks.workunit.client.0.smithi096.stdout:Building tmpfs 2024-04-25T21:00:33.762 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-04-25T21:00:33.768 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-04-25T21:00:33.780 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dataascii.lo 2024-04-25T21:00:33.781 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] databin.lo 2024-04-25T21:00:33.783 INFO:tasks.workunit.client.0.smithi096.stdout:Building udf 2024-04-25T21:00:33.790 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-04-25T21:00:33.803 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] datapid.lo 2024-04-25T21:00:33.857 INFO:tasks.workunit.client.0.smithi096.stdout:Building xfs 2024-04-25T21:00:33.957 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] file_lock.lo 2024-04-25T21:00:33.958 INFO:tasks.workunit.client.0.smithi096.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-04-25T21:00:33.962 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] forker.lo 2024-04-25T21:00:33.977 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] pattern.lo 2024-04-25T21:00:34.080 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] open_flags.lo 2024-04-25T21:00:34.118 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] random_range.lo 2024-04-25T21:00:34.139 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] string_to_tokens.lo 2024-04-25T21:00:34.155 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] str_to_bytes.lo 2024-04-25T21:00:34.263 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] write_log.lo 2024-04-25T21:00:34.264 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] tlibio.lo 2024-04-25T21:00:34.327 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] random.lo 2024-04-25T21:00:34.537 INFO:tasks.workunit.client.0.smithi096.stdout: [LD] 2024-04-25T21:00:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:34 smithi097 ceph-mon[32667]: pgmap v91: 97 pgs: 97 active+clean; 30 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 967 op/s 2024-04-25T21:00:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:34 smithi139 ceph-mon[33027]: pgmap v91: 97 pgs: 97 active+clean; 30 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 967 op/s 2024-04-25T21:00:34.649 INFO:tasks.workunit.client.0.smithi096.stdout:Building ltp 2024-04-25T21:00:34.650 INFO:tasks.workunit.client.0.smithi096.stdout:Building src 2024-04-25T21:00:34.650 INFO:tasks.workunit.client.0.smithi096.stdout:Building m4 2024-04-25T21:00:34.650 INFO:tasks.workunit.client.0.smithi096.stdout:Building common 2024-04-25T21:00:34.655 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T21:00:34.666 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T21:00:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:34 smithi096 ceph-mon[29288]: pgmap v91: 97 pgs: 97 active+clean; 30 MiB data, 551 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.7 MiB/s wr, 967 op/s 2024-04-25T21:00:34.855 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] doio 2024-04-25T21:00:34.855 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsstress 2024-04-25T21:00:34.855 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsx 2024-04-25T21:00:35.526 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] growfiles 2024-04-25T21:00:35.597 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] iogen 2024-04-25T21:00:35.962 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-stress 2024-04-25T21:00:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:36 smithi097 ceph-mon[32667]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 602 op/s 2024-04-25T21:00:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:36 smithi139 ceph-mon[33027]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 602 op/s 2024-04-25T21:00:36.665 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T21:00:36.666 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T21:00:36.667 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T21:00:36.669 INFO:tasks.workunit.client.0.smithi096.stdout: [DEP] 2024-04-25T21:00:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:36 smithi096 ceph-mon[29288]: pgmap v92: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 602 op/s 2024-04-25T21:00:36.904 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dirstress 2024-04-25T21:00:36.904 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fill 2024-04-25T21:00:36.904 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fill2 2024-04-25T21:00:36.904 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] getpagesize 2024-04-25T21:00:37.067 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] lstat64 2024-04-25T21:00:37.067 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] holes 2024-04-25T21:00:37.162 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] nametest 2024-04-25T21:00:37.163 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] permname 2024-04-25T21:00:37.265 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] randholes 2024-04-25T21:00:37.284 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] runas 2024-04-25T21:00:37.401 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] truncfile 2024-04-25T21:00:37.450 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mmapcat 2024-04-25T21:00:37.450 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] usemem 2024-04-25T21:00:37.522 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] append_reader 2024-04-25T21:00:37.579 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] append_writer 2024-04-25T21:00:37.606 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dirperf 2024-04-25T21:00:37.615 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] metaperf 2024-04-25T21:00:37.673 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] devzero 2024-04-25T21:00:37.738 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] feature 2024-04-25T21:00:37.841 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] alloc 2024-04-25T21:00:37.857 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fault 2024-04-25T21:00:37.903 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fstest 2024-04-25T21:00:37.956 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_access_root 2024-04-25T21:00:38.016 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] godown 2024-04-25T21:00:38.075 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] resvtest 2024-04-25T21:00:38.128 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] writemod 2024-04-25T21:00:38.149 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] writev_on_pagefault 2024-04-25T21:00:38.194 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] makeextents 2024-04-25T21:00:38.280 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] itrash 2024-04-25T21:00:38.286 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] rename 2024-04-25T21:00:38.335 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] multi_open_unlink 2024-04-25T21:00:38.392 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] unwritten_sync 2024-04-25T21:00:38.423 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] genhashnames 2024-04-25T21:00:38.434 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_holes 2024-04-25T21:00:38.505 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_writev 2024-04-25T21:00:38.572 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_truncate_cmtime 2024-04-25T21:00:38.589 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dirhash_collide 2024-04-25T21:00:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:38 smithi097 ceph-mon[32667]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 586 op/s 2024-04-25T21:00:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:38 smithi139 ceph-mon[33027]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 586 op/s 2024-04-25T21:00:38.631 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_rename_overwrite 2024-04-25T21:00:38.666 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] holetest 2024-04-25T21:00:38.740 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_truncate_self 2024-04-25T21:00:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:38 smithi096 ceph-mon[29288]: pgmap v93: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.4 MiB/s wr, 586 op/s 2024-04-25T21:00:38.789 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] af_unix 2024-04-25T21:00:38.790 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_stale_pmd 2024-04-25T21:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 -- 172.21.15.96:0/1693155257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838101a80 msgr2=0x7f4838105ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 --2- 172.21.15.96:0/1693155257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838101a80 0x7f4838105ac0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f482c009860 tx=0x7f482c02a340 comp rx=0 tx=0).stop 2024-04-25T21:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 -- 172.21.15.96:0/1693155257 shutdown_connections 2024-04-25T21:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 --2- 172.21.15.96:0/1693155257 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 0x7f483810dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 --2- 172.21.15.96:0/1693155257 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838101a80 0x7f4838105ac0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.833 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 --2- 172.21.15.96:0/1693155257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48381010d0 0x7f48381014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 -- 172.21.15.96:0/1693155257 >> 172.21.15.96:0/1693155257 conn(0x7f4838078060 msgr2=0x7f4838078470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 -- 172.21.15.96:0/1693155257 shutdown_connections 2024-04-25T21:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 -- 172.21.15.96:0/1693155257 wait complete. 2024-04-25T21:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 Processor -- start 2024-04-25T21:00:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.832+0000 7f483e267640 1 -- start start 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483e267640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 0x7f483810e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483e267640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 0x7f483810ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483e267640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f483810f180 0x7f48381a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483e267640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4838112220 con 0x7f483810f180 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483e267640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48381120a0 con 0x7f48381010d0 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483e267640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48381123a0 con 0x7f48381061a0 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 0x7f483810e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f48377fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 0x7f483810ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 0x7f483810e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:46728/0 (socket says 172.21.15.96:46728) 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 -- 172.21.15.96:0/2630416909 learned_addr learned my addr 172.21.15.96:0/2630416909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f483c804640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f483810f180 0x7f48381a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 -- 172.21.15.96:0/2630416909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 msgr2=0x7f483810ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 0x7f483810ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 -- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f483810f180 msgr2=0x7f48381a68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.833+0000 7f4837fff640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f483810f180 0x7f48381a68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f4837fff640 1 -- 172.21.15.96:0/2630416909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48381a6f50 con 0x7f48381010d0 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f48377fe640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 0x7f483810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f483c804640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f483810f180 0x7f48381a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f4837fff640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 0x7f483810e700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f482800e9c0 tx=0x7f482800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f48357fa640 1 -- 172.21.15.96:0/2630416909 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482800cde0 con 0x7f48381010d0 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f48357fa640 1 -- 172.21.15.96:0/2630416909 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48280035f0 con 0x7f48381010d0 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.834+0000 7f48357fa640 1 -- 172.21.15.96:0/2630416909 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828010690 con 0x7f48381010d0 2024-04-25T21:00:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.835+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381a7240 con 0x7f48381010d0 2024-04-25T21:00:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.835+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48381a7710 con 0x7f48381010d0 2024-04-25T21:00:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.838+0000 7f48357fa640 1 -- 172.21.15.96:0/2630416909 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4828004000 con 0x7f48381010d0 2024-04-25T21:00:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.838+0000 7f48357fa640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48200a3e40 0x7f48200a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.840 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f48357fa640 1 -- 172.21.15.96:0/2630416909 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f48280d0aa0 con 0x7f48381010d0 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 Processor -- start 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f48377fe640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48200a3e40 0x7f48200a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 -- start start 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 0x558f78e68920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e68e60 0x558f78e7bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e7c010 0x558f78e7e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f78dcd970 con 0x558f78e68540 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558f78d6a4a0 con 0x558f78e68e60 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4840f78740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558f78dc9930 con 0x558f78e7c010 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4837fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 0x558f78e68920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f48377fe640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48200a3e40 0x7f48200a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f482c0303f0 tx=0x7f482c030340 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:38.841 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4837fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 0x558f78e68920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38778/0 (socket says 172.21.15.96:38778) 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.839+0000 7f4837fff640 1 -- 172.21.15.96:0/2946298114 learned_addr learned my addr 172.21.15.96:0/2946298114 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f48377fe640 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e68e60 0x558f78e7bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f483c804640 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e7c010 0x558f78e7e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f4837fff640 1 -- 172.21.15.96:0/2946298114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e7c010 msgr2=0x558f78e7e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f4837fff640 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e7c010 0x558f78e7e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f4837fff640 1 -- 172.21.15.96:0/2946298114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e68e60 msgr2=0x558f78e7bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f4837fff640 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e68e60 0x558f78e7bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f4837fff640 1 -- 172.21.15.96:0/2946298114 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f78ae0210 con 0x558f78e68540 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f48377fe640 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e68e60 0x558f78e7bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:00:38.842 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f4837fff640 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 0x558f78e68920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4828099480 tx=0x7f48280042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b66191b8a6a7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f481e7fc640 1 -- 172.21.15.96:0/2946298114 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482809cab0 con 0x558f78e68540 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f481e7fc640 1 -- 172.21.15.96:0/2946298114 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48280a0e60 con 0x558f78e68540 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.840+0000 7f481e7fc640 1 -- 172.21.15.96:0/2946298114 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280ce4e0 con 0x558f78e68540 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.841+0000 7f4840f78740 1 -- 172.21.15.96:0/2946298114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 msgr2=0x558f78e68920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.841+0000 7f4840f78740 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 0x558f78e68920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f4828099480 tx=0x7f48280042f0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 -- 172.21.15.96:0/2946298114 shutdown_connections 2024-04-25T21:00:38.846 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e7c010 0x558f78e7e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e68e60 0x558f78e7bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 --2- 172.21.15.96:0/2946298114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68540 0x558f78e68920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 -- 172.21.15.96:0/2946298114 >> 172.21.15.96:0/2946298114 conn(0x558f78e64b30 msgr2=0x558f78e64f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 -- 172.21.15.96:0/2946298114 shutdown_connections 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 -- 172.21.15.96:0/2946298114 wait complete. 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 -- start start 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.842+0000 7f4840f78740 1 Processor -- start 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 0x558f78e78f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e79490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e799d0 0x558f78e88070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558f78e7ec40 con 0x558f78e68e60 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x558f78d680b0 con 0x558f78e7c010 2024-04-25T21:00:38.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x558f78e7edc0 con 0x558f78e799d0 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4840f78740 10 client.0 _ll_register_callbacks cb 0x558f78d55d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 0x558f78e78f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 0x558f78e78f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38792/0 (socket says 172.21.15.96:38792) 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 -- 172.21.15.96:0/1383677134 learned_addr learned my addr 172.21.15.96:0/1383677134 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f48377fe640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e79490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f483c804640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e799d0 0x558f78e88070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 -- 172.21.15.96:0/1383677134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e799d0 msgr2=0x558f78e88070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e799d0 0x558f78e88070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 -- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 msgr2=0x558f78e79490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e79490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 -- 172.21.15.96:0/1383677134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f78bac620 con 0x558f78e68e60 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.843+0000 7f4837fff640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 0x558f78e78f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f482809ca40 tx=0x7f48280a5870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a15ef85c3ea31bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:38.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f48177fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f48177fe640 1 -- 172.21.15.96:0/1383677134 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280037a0 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f48177fe640 1 -- 172.21.15.96:0/1383677134 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f4828099120 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48280a0e60 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f48177fe640 1 -- 172.21.15.96:0/1383677134 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f482809bf00 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f4840f78740 10 client.15663 fetch_fsmap learned FSMap version 29 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f78e8b220 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f48177fe640 1 -- 172.21.15.96:0/1383677134 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48280a0e60 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.844+0000 7f48177fe640 1 -- 172.21.15.96:0/1383677134 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f482809bb10 con 0x558f78e68e60 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f4840f78740 10 client.15663 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f4840f78740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f4840f78740 20 client.15663 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f4840f78740 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78e8c570 0x558f78e8e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f4840f78740 4 client.15663 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:38.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f78ddb390 con 0x558f78e8c570 2024-04-25T21:00:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f48377fe640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78e8c570 0x558f78e8e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f48377fe640 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78e8c570 0x558f78e8e960 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76e22492164b191 server_cookie=397c0be717f36409 in_seq=0 out_seq=0 2024-04-25T21:00:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.845+0000 7f48177fe640 10 client.15663 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.846+0000 7f48177fe640 1 -- 172.21.15.96:0/1383677134 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558f78ddb390 con 0x558f78e8c570 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.846+0000 7f48177fe640 10 client.15663 handle_command_reply: tid=0 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.846+0000 7f4840f78740 1 client.15663 shutdown 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.846+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 msgr2=0x558f78e78f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.846+0000 7f4840f78740 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 0x558f78e78f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f482809ca40 tx=0x7f48280a5870 comp rx=0 tx=0).stop 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.846+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 shutdown_connections 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78e8c570 0x558f78e8e960 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e799d0 0x558f78e88070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e79490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 --2- 172.21.15.96:0/1383677134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e68e60 0x558f78e78f50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 >> 172.21.15.96:0/1383677134 conn(0x558f78e64b30 msgr2=0x558f78e6a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f48177fe640 0 client.15663 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 shutdown_connections 2024-04-25T21:00:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 1 -- 172.21.15.96:0/1383677134 wait complete. 2024-04-25T21:00:38.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.847+0000 7f4840f78740 20 client.15663 trim_cache size 0 max 16384 2024-04-25T21:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.871+0000 7f4840f78740 1 Processor -- start 2024-04-25T21:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.871+0000 7f4840f78740 1 -- start start 2024-04-25T21:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4840f78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e799d0 0x558f78e88020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4840f78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 0x558f78e64430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4840f78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e91de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4840f78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ae150 con 0x558f78e7c010 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4840f78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ab9c0 con 0x558f78e799d0 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4840f78740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48381a9870 con 0x558f78e91980 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f48377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 0x558f78e64430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e799d0 0x558f78e88020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f48377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 0x558f78e64430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38808/0 (socket says 172.21.15.96:38808) 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f48377fe640 1 -- 172.21.15.96:0/2349168761 learned_addr learned my addr 172.21.15.96:0/2349168761 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f483c804640 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e91de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 -- 172.21.15.96:0/2349168761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 msgr2=0x558f78e91de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e91de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 -- 172.21.15.96:0/2349168761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 msgr2=0x558f78e64430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 0x558f78e64430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 -- 172.21.15.96:0/2349168761 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f78e7ec90 con 0x558f78e799d0 2024-04-25T21:00:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f48377fe640 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 0x558f78e64430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:00:38.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f4837fff640 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e799d0 0x558f78e88020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4828011000 tx=0x7f48280bd050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600691a41e6eb026 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.872+0000 7f481dffb640 1 -- 172.21.15.96:0/2349168761 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48280a2380 con 0x558f78e799d0 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f481dffb640 1 -- 172.21.15.96:0/2349168761 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48280a0e60 con 0x558f78e799d0 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f481dffb640 1 -- 172.21.15.96:0/2349168761 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828099120 con 0x558f78e799d0 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 -- 172.21.15.96:0/2349168761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e799d0 msgr2=0x558f78e88020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e799d0 0x558f78e88020 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4828011000 tx=0x7f48280bd050 comp rx=0 tx=0).stop 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 -- 172.21.15.96:0/2349168761 shutdown_connections 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e91de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e7c010 0x558f78e64430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 --2- 172.21.15.96:0/2349168761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e799d0 0x558f78e88020 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 -- 172.21.15.96:0/2349168761 >> 172.21.15.96:0/2349168761 conn(0x558f78e64b30 msgr2=0x558f78e74300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 -- 172.21.15.96:0/2349168761 shutdown_connections 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.873+0000 7f4840f78740 1 -- 172.21.15.96:0/2349168761 wait complete. 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 -- start start 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 Processor -- start 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e799d0 0x558f78e8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e8f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48381a9ec0 con 0x558f78e799d0 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ab2f0 con 0x558f78e7c010 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f48381aa550 con 0x558f78e91980 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.874+0000 7f4840f78740 10 client.0 _ll_register_callbacks cb 0x7f48280a5720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e8f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e8f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:60952/0 (socket says 172.21.15.96:60952) 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 -- 172.21.15.96:0/454906130 learned_addr learned my addr 172.21.15.96:0/454906130 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f4837fff640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e799d0 0x558f78e8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f48377fe640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 -- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 msgr2=0x558f78e8f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e8f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 -- 172.21.15.96:0/454906130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e799d0 msgr2=0x558f78e8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e799d0 0x558f78e8ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 -- 172.21.15.96:0/454906130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f78ef1950 con 0x558f78e91980 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f483c804640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e8f7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4830011d10 tx=0x7f48300107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b40101e28ca87e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.875+0000 7f48177fe640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f48177fe640 1 -- 172.21.15.96:0/454906130 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830004b20 con 0x558f78e91980 2024-04-25T21:00:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f48177fe640 1 -- 172.21.15.96:0/454906130 <== mon.1 v2:172.21.15.139:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f4830004cc0 con 0x558f78e91980 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f48280a0e60 con 0x558f78e91980 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f48177fe640 1 -- 172.21.15.96:0/454906130 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4830005020 con 0x558f78e91980 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 10 client.25111 fetch_fsmap learned FSMap version 29 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558f78ef2ca0 con 0x558f78e91980 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f48177fe640 1 -- 172.21.15.96:0/454906130 <== mon.1 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48280a0e60 con 0x558f78e91980 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f48177fe640 1 -- 172.21.15.96:0/454906130 <== mon.1 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f4830025480 con 0x558f78e91980 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 10 client.25111 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 10 client.25111 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 20 client.25111 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.876+0000 7f4840f78740 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78ef34d0 0x558f78ef58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.877+0000 7f4840f78740 4 client.25111 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.877+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558f78dd5200 con 0x558f78ef34d0 2024-04-25T21:00:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.877+0000 7f4837fff640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78ef34d0 0x558f78ef58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.877+0000 7f48177fe640 10 client.25111 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.877+0000 7f4837fff640 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78ef34d0 0x558f78ef58f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1217276013d7d5f5 server_cookie=63891e06f001afb8 in_seq=0 out_seq=0 2024-04-25T21:00:38.886 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_cow_race 2024-04-25T21:00:38.982 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f48177fe640 1 -- 172.21.15.96:0/454906130 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558f78dd5200 con 0x558f78ef34d0 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f48177fe640 10 client.25111 handle_command_reply: tid=0 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 client.25111 shutdown 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 msgr2=0x558f78e8f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e8f7f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4830011d10 tx=0x7f48300107a0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 shutdown_connections 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x558f78ef34d0 0x558f78ef58f0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x558f78e91980 0x558f78e8f7f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f78e7c010 0x558f78e8f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f48177fe640 0 client.25111 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 --2- 172.21.15.96:0/454906130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558f78e799d0 0x558f78e8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.981+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 >> 172.21.15.96:0/454906130 conn(0x558f78e64b30 msgr2=0x558f78e6a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.982+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 shutdown_connections 2024-04-25T21:00:38.983 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.982+0000 7f4840f78740 1 -- 172.21.15.96:0/454906130 wait complete. 2024-04-25T21:00:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.982+0000 7f4840f78740 20 client.25111 trim_cache size 0 max 16384 2024-04-25T21:00:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:00:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:00:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:00:38.984 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48200a3e40 msgr2=0x7f48200a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48200a3e40 0x7f48200a6300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f482c0303f0 tx=0x7f482c030340 comp rx=0 tx=0).stop 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 msgr2=0x7f483810e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 0x7f483810e700 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f482800e9c0 tx=0x7f482800ee80 comp rx=0 tx=0).stop 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 shutdown_connections 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f48200a3e40 0x7f48200a6300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f483810f180 0x7f48381a68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.985 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f48381061a0 0x7f483810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 --2- 172.21.15.96:0/2630416909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f48381010d0 0x7f483810e700 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 >> 172.21.15.96:0/2630416909 conn(0x7f4838078060 msgr2=0x7f48380fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 shutdown_connections 2024-04-25T21:00:38.986 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:38.983+0000 7f483e267640 1 -- 172.21.15.96:0/2630416909 wait complete. 2024-04-25T21:00:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:40 smithi097 ceph-mon[32667]: pgmap v94: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 501 op/s 2024-04-25T21:00:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:40 smithi139 ceph-mon[33027]: pgmap v94: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 501 op/s 2024-04-25T21:00:40.678 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_fallocate 2024-04-25T21:00:40.739 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsync-err 2024-04-25T21:00:40.744 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_write_ro 2024-04-25T21:00:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:40 smithi096 ceph-mon[29288]: pgmap v94: 97 pgs: 97 active+clean; 33 MiB data, 549 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 501 op/s 2024-04-25T21:00:40.833 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_ext4_dax_journal_corruption 2024-04-25T21:00:40.861 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_ext4_dax_inline_corruption 2024-04-25T21:00:40.901 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_ofd_locks 2024-04-25T21:00:40.948 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_collision 2024-04-25T21:00:41.003 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mmap-write-concurrent 2024-04-25T21:00:41.032 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_get_file_time 2024-04-25T21:00:41.108 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_create_short_dirs 2024-04-25T21:00:41.178 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_enospc 2024-04-25T21:00:41.178 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_create_long_dirs 2024-04-25T21:00:41.183 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_writev_overlap 2024-04-25T21:00:41.290 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] checkpoint_journal 2024-04-25T21:00:41.404 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] allocstale 2024-04-25T21:00:41.404 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mmap-rw-fault 2024-04-25T21:00:41.458 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_cow_memory_failure 2024-04-25T21:00:41.477 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fake-dump-rootino 2024-04-25T21:00:41.578 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dio-buf-fault 2024-04-25T21:00:41.584 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] rewinddir-test 2024-04-25T21:00:41.630 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] readdir-while-renames 2024-04-25T21:00:41.642 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] xfsctl 2024-04-25T21:00:41.728 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bstat 2024-04-25T21:00:41.745 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mtab 2024-04-25T21:00:41.795 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] getdevicesize 2024-04-25T21:00:41.839 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] preallo_rw_pattern_reader 2024-04-25T21:00:41.932 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] preallo_rw_pattern_writer 2024-04-25T21:00:41.937 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] ftrunc 2024-04-25T21:00:41.942 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] trunc 2024-04-25T21:00:42.006 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fs_perms 2024-04-25T21:00:42.094 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] testx 2024-04-25T21:00:42.103 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] looptest 2024-04-25T21:00:42.105 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] locktest 2024-04-25T21:00:42.169 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] unwritten_mmap 2024-04-25T21:00:42.223 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bulkstat_unlink_test 2024-04-25T21:00:42.322 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] deduperace 2024-04-25T21:00:42.340 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bulkstat_unlink_test_modified 2024-04-25T21:00:42.436 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_dir_offset 2024-04-25T21:00:42.454 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_futimens 2024-04-25T21:00:42.537 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_immutable 2024-04-25T21:00:42.563 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] stale_handle 2024-04-25T21:00:42.605 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] pwrite_mmap_blocked 2024-04-25T21:00:42.611 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_dir_offset2 2024-04-25T21:00:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:42 smithi097 ceph-mon[32667]: pgmap v95: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 617 op/s 2024-04-25T21:00:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:42 smithi139 ceph-mon[33027]: pgmap v95: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 617 op/s 2024-04-25T21:00:42.729 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] seek_sanity_test 2024-04-25T21:00:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:42 smithi096 ceph-mon[29288]: pgmap v95: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 617 op/s 2024-04-25T21:00:42.750 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] seek_copy_test 2024-04-25T21:00:42.791 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_readdir_1 2024-04-25T21:00:43.253 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_readdir_2 2024-04-25T21:00:43.277 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fsync-tester 2024-04-25T21:00:43.301 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] nsexec 2024-04-25T21:00:43.308 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] cloner 2024-04-25T21:00:43.418 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] renameat2 2024-04-25T21:00:43.475 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_getcwd 2024-04-25T21:00:43.495 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] e4compact 2024-04-25T21:00:43.528 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] test-nextquota 2024-04-25T21:00:43.595 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] punch-alternating 2024-04-25T21:00:43.663 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] attr-list-by-handle-cursor-test 2024-04-25T21:00:43.742 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] listxattr 2024-04-25T21:00:43.782 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dio-interleaved 2024-04-25T21:00:43.813 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_dir_type 2024-04-25T21:00:43.875 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] dio-invalidate-cache 2024-04-25T21:00:43.886 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] stat_test 2024-04-25T21:00:43.952 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_encrypted_d_revalidate 2024-04-25T21:00:43.968 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] attr_replace_test 2024-04-25T21:00:44.079 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] swapon 2024-04-25T21:00:44.118 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mkswap 2024-04-25T21:00:44.135 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_attr_corruption 2024-04-25T21:00:44.191 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_open_tmpfiles 2024-04-25T21:00:44.240 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fscrypt-crypt-util 2024-04-25T21:00:44.282 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] bulkstat_null_ocount 2024-04-25T21:00:44.286 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] splice-test 2024-04-25T21:00:44.372 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] chprojid_fail 2024-04-25T21:00:44.453 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] detached_mounts_propagation 2024-04-25T21:00:44.461 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] ext4_resize 2024-04-25T21:00:44.540 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_readdir_3 2024-04-25T21:00:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:44 smithi097 ceph-mon[32667]: pgmap v96: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 347 KiB/s wr, 149 op/s 2024-04-25T21:00:44.613 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] splice2pipe 2024-04-25T21:00:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:44 smithi139 ceph-mon[33027]: pgmap v96: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 347 KiB/s wr, 149 op/s 2024-04-25T21:00:44.644 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] uuid_ioctl 2024-04-25T21:00:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:44 smithi096 ceph-mon[29288]: pgmap v96: 97 pgs: 97 active+clean; 29 MiB data, 550 MiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 347 KiB/s wr, 149 op/s 2024-04-25T21:00:44.750 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_snapshot_deleted_subvolume 2024-04-25T21:00:44.774 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fiemap-fault 2024-04-25T21:00:44.822 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fiemap-tester 2024-04-25T21:00:44.923 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_stripealign 2024-04-25T21:00:44.941 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] open_by_handle 2024-04-25T21:00:44.960 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_mmap_dio 2024-04-25T21:00:45.075 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] xfsfind 2024-04-25T21:00:45.102 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] t_reflink_read_race 2024-04-25T21:00:45.136 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] fssum 2024-04-25T21:00:45.208 INFO:tasks.workunit.client.0.smithi096.stdout:Building log-writes 2024-04-25T21:00:45.214 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] replay-log 2024-04-25T21:00:45.274 INFO:tasks.workunit.client.0.smithi096.stdout:Building perf 2024-04-25T21:00:45.278 INFO:tasks.workunit.client.0.smithi096.stdout:Building aio-dio-regress 2024-04-25T21:00:45.286 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aiocp 2024-04-25T21:00:45.305 INFO:tasks.workunit.client.0.smithi096.stdout:Building vfs 2024-04-25T21:00:45.312 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] vfstest 2024-04-25T21:00:45.514 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-append-write-fallocate-race 2024-04-25T21:00:45.541 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-append-write-read-race 2024-04-25T21:00:45.598 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-cow-race 2024-04-25T21:00:45.692 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-cycle-write 2024-04-25T21:00:45.705 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-eof-race 2024-04-25T21:00:45.752 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-extend-stat 2024-04-25T21:00:45.874 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-fcntl-race 2024-04-25T21:00:45.885 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-hole-filling-race 2024-04-25T21:00:45.920 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-invalidate-failure 2024-04-25T21:00:46.022 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-invalidate-readahead 2024-04-25T21:00:46.027 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aiodio_sparse2 2024-04-25T21:00:46.091 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] mount-idmapped 2024-04-25T21:00:46.213 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-subblock-eof-read 2024-04-25T21:00:46.266 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-dio-write-verify 2024-04-25T21:00:46.394 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-04-25T21:00:46.465 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-04-25T21:00:46.557 INFO:tasks.workunit.client.0.smithi096.stdout: [CC] aio-last-ref-held-by-io 2024-04-25T21:00:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:46 smithi097 ceph-mon[32667]: pgmap v97: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-04-25T21:00:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:46 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:46 smithi139 ceph-mon[33027]: pgmap v97: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-04-25T21:00:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:46 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:46 smithi096 ceph-mon[29288]: pgmap v97: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 602 KiB/s rd, 1.0 MiB/s wr, 242 op/s 2024-04-25T21:00:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:46 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:00:47.410 INFO:tasks.workunit.client.0.smithi096.stderr:+ cd .. 2024-04-25T21:00:47.411 INFO:tasks.workunit.client.0.smithi096.stderr:+ cp xfstests-dev/ltp/fsx . 2024-04-25T21:00:47.414 INFO:tasks.workunit.client.0.smithi096.stderr:+ OPTIONS=-z 2024-04-25T21:00:47.414 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-04-25T21:00:47.414 INFO:tasks.workunit.client.0.smithi096.stdout:Seed set to 1 2024-04-25T21:00:47.452 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T21:00:47.452 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T21:00:47.455 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T21:00:47.455 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T21:00:47.455 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T21:00:47.458 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T21:00:47.461 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T21:00:47.461 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T21:00:47.461 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T21:00:47.461 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x81d54 2024-04-25T21:00:47.467 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xb86d7 2024-04-25T21:00:47.474 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xe627e 2024-04-25T21:00:47.514 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0xf3194 2024-04-25T21:00:47.536 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xf3ceb 2024-04-25T21:00:47.542 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0xf47e4 2024-04-25T21:00:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:47 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:47 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:47.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:47 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:00:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:47 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:47 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:47.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:47 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:00:47.631 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xfaccc 2024-04-25T21:00:47.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:47 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:00:47.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:47 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:00:47.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:47 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:00:47.991 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xfe423 2024-04-25T21:00:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:48 smithi097 ceph-mon[32667]: pgmap v98: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 747 KiB/s wr, 208 op/s 2024-04-25T21:00:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:48 smithi139 ceph-mon[33027]: pgmap v98: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 747 KiB/s wr, 208 op/s 2024-04-25T21:00:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:48 smithi096 ceph-mon[29288]: pgmap v98: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 747 KiB/s wr, 208 op/s 2024-04-25T21:00:50.136 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xfe492 2024-04-25T21:00:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:50 smithi097 ceph-mon[32667]: pgmap v99: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 747 KiB/s wr, 208 op/s 2024-04-25T21:00:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:50 smithi139 ceph-mon[33027]: pgmap v99: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 747 KiB/s wr, 208 op/s 2024-04-25T21:00:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:50 smithi096 ceph-mon[29288]: pgmap v99: 97 pgs: 97 active+clean; 35 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 747 KiB/s wr, 208 op/s 2024-04-25T21:00:52.006 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xff038 2024-04-25T21:00:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 -- 172.21.15.96:0/547838501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103200 msgr2=0x7f1ed41035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.079 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 --2- 172.21.15.96:0/547838501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103200 0x7f1ed41035e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009ca0 tx=0x7f1ec402a950 comp rx=0 tx=0).stop 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 -- 172.21.15.96:0/547838501 shutdown_connections 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 --2- 172.21.15.96:0/547838501 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ed410c5d0 0x7f1ed410ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 --2- 172.21.15.96:0/547838501 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed4103b20 0x7f1ed410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 --2- 172.21.15.96:0/547838501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103200 0x7f1ed41035e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 -- 172.21.15.96:0/547838501 >> 172.21.15.96:0/547838501 conn(0x7f1ed40771d0 msgr2=0x7f1ed40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.076+0000 7f1ed948a640 1 -- 172.21.15.96:0/547838501 shutdown_connections 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 -- 172.21.15.96:0/547838501 wait complete. 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 Processor -- start 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 -- start start 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed4103200 0x7f1ed41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 0x7f1ed4198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.080 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ed410c5d0 0x7f1ed419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4112200 con 0x7f1ed4103b20 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4112080 con 0x7f1ed4103200 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.077+0000 7f1ed948a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4112380 con 0x7f1ed410c5d0 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 0x7f1ed4198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 0x7f1ed4198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39674/0 (socket says 172.21.15.96:39674) 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed4103200 0x7f1ed41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 -- 172.21.15.96:0/555126969 learned_addr learned my addr 172.21.15.96:0/555126969 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ed410c5d0 0x7f1ed419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 -- 172.21.15.96:0/555126969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ed410c5d0 msgr2=0x7f1ed419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ed410c5d0 0x7f1ed419cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 -- 172.21.15.96:0/555126969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed4103200 msgr2=0x7f1ed41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed4103200 0x7f1ed41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.081 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 -- 172.21.15.96:0/555126969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed419d6b0 con 0x7f1ed4103b20 2024-04-25T21:00:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed27fc640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 0x7f1ed4198c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e9f0 tx=0x7f1ec800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ebbfff640 1 -- 172.21.15.96:0/555126969 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec800cd80 con 0x7f1ed4103b20 2024-04-25T21:00:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed419d9a0 con 0x7f1ed4103b20 2024-04-25T21:00:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ebbfff640 1 -- 172.21.15.96:0/555126969 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec80108e0 con 0x7f1ed4103b20 2024-04-25T21:00:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ebbfff640 1 -- 172.21.15.96:0/555126969 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec8018980 con 0x7f1ed4103b20 2024-04-25T21:00:52.082 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.078+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41a5230 con 0x7f1ed4103b20 2024-04-25T21:00:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.080+0000 7f1ebbfff640 1 -- 172.21.15.96:0/555126969 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ec8010430 con 0x7f1ed4103b20 2024-04-25T21:00:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 Processor -- start 2024-04-25T21:00:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 -- start start 2024-04-25T21:00:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ebbfff640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1ebc0a3dc0 0x7f1ebc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 0x55c4a5ce0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.088 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ceb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5cedbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5c45970 con 0x55c4a5ceb800 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5be24a0 con 0x55c4a5ce0620 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5c41930 con 0x55c4a5ce0f40 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 0x55c4a5ce0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 0x55c4a5ce0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:47104/0 (socket says 172.21.15.96:47104) 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/3310585416 learned_addr learned my addr 172.21.15.96:0/3310585416 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5cedbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ed27fc640 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ceb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ebbfff640 1 -- 172.21.15.96:0/555126969 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f1ec80d0ab0 con 0x7f1ed4103b20 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.081+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1ebc0a3dc0 0x7f1ebc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/3310585416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 msgr2=0x55c4a5ceb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ceb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.089 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/3310585416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 msgr2=0x55c4a5cedbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5cedbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/3310585416 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a5958210 con 0x55c4a5ce0620 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 0x55c4a5ce0a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009ca0 tx=0x7f1ec4005f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7792b4e1a2f9b23a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1ebc0a3dc0 0x7f1ebc0a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55c4a5cddf50 tx=0x7f1ecc005f90 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1eb97fa640 1 -- 172.21.15.96:0/3310585416 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4048070 con 0x55c4a5ce0620 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1eb97fa640 1 -- 172.21.15.96:0/3310585416 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4003e20 con 0x55c4a5ce0620 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.082+0000 7f1eb97fa640 1 -- 172.21.15.96:0/3310585416 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4034040 con 0x55c4a5ce0620 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- 172.21.15.96:0/3310585416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 msgr2=0x55c4a5ce0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 0x55c4a5ce0a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009ca0 tx=0x7f1ec4005f50 comp rx=0 tx=0).stop 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- 172.21.15.96:0/3310585416 shutdown_connections 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5cedbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ceb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.090 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/3310585416 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0620 0x55c4a5ce0a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- 172.21.15.96:0/3310585416 >> 172.21.15.96:0/3310585416 conn(0x55c4a5cdcb30 msgr2=0x55c4a5cdcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- 172.21.15.96:0/3310585416 shutdown_connections 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- 172.21.15.96:0/3310585416 wait complete. 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- start start 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 Processor -- start 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5ceb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ce8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5ce89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5cee3d0 con 0x55c4a5ceb800 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5be00b0 con 0x55c4a5ce0f40 2024-04-25T21:00:52.091 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55c4a5cee550 con 0x55c4a5ce0620 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.083+0000 7f1edc1a7740 10 client.0 _ll_register_callbacks cb 0x55c4a5bcdd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5ce89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5ceb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5ce89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39698/0 (socket says 172.21.15.96:39698) 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5ceb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:36578/0 (socket says 172.21.15.96:36578) 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 -- 172.21.15.96:0/1334920633 learned_addr learned my addr 172.21.15.96:0/1334920633 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed27fc640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ce8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 -- 172.21.15.96:0/1334920633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 msgr2=0x55c4a5ceb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5ceb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 -- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 msgr2=0x55c4a5ce8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ce8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 -- 172.21.15.96:0/1334920633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a5a24620 con 0x55c4a5ceb800 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5ce89a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc011e30 tx=0x7f1ecc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e3f83a31ee14ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.092 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1eb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/1334920633 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc0044e0 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/1334920633 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f1ecc011220 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec4003e20 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/1334920633 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ecc0035f0 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1edc1a7740 10 client.15681 fetch_fsmap learned FSMap version 29 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4a5cfa3f0 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.084+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/1334920633 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4003e20 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/1334920633 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1ecc032050 con 0x55c4a5ceb800 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1edc1a7740 10 client.15681 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1edc1a7740 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1edc1a7740 20 client.15681 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5cfb920 0x55c4a5cfdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1edc1a7740 4 client.15681 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:00:52.093 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a5c53390 con 0x55c4a5cfb920 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5cfb920 0x55c4a5cfdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1eb6ff5640 10 client.15681 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.085+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5cfb920 0x55c4a5cfdd50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee31665021a64996 server_cookie=1b36156f0a9a5ca6 in_seq=0 out_seq=0 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/1334920633 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4a5c53390 con 0x55c4a5cfb920 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1eb6ff5640 10 client.15681 handle_command_reply: tid=0 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 client.15681 shutdown 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 msgr2=0x55c4a5ce89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5ce89a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc011e30 tx=0x7f1ecc004730 comp rx=0 tx=0).stop 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 shutdown_connections 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5cfb920 0x55c4a5cfdd50 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5ce89a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5ce8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/1334920633 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5ceb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.094 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 >> 172.21.15.96:0/1334920633 conn(0x55c4a5cdcb30 msgr2=0x55c4a5ce2860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1eb6ff5640 0 client.15681 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 shutdown_connections 2024-04-25T21:00:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 1 -- 172.21.15.96:0/1334920633 wait complete. 2024-04-25T21:00:52.095 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.086+0000 7f1edc1a7740 20 client.15681 trim_cache size 0 max 16384 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 Processor -- start 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 -- start start 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cdbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ceb800 0x55c4a5d03660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41b02c0 con 0x55c4a5ce0620 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41adb30 con 0x55c4a5ceb800 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ab980 con 0x55c4a5ce0f40 2024-04-25T21:00:52.106 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39708/0 (socket says 172.21.15.96:39708) 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/749843086 learned_addr learned my addr 172.21.15.96:0/749843086 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ceb800 0x55c4a5d03660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed27fc640 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cdbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/749843086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 msgr2=0x55c4a5cdbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cdbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/749843086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ceb800 msgr2=0x55c4a5d03660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ceb800 0x55c4a5d03660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.101+0000 7f1ed2ffd640 1 -- 172.21.15.96:0/749843086 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a5cd82e0 con 0x55c4a5ce0620 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf7290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009bf0 tx=0x7f1ec402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be5519d690e94e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1eb8ff9640 1 -- 172.21.15.96:0/749843086 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4048040 con 0x55c4a5ce0620 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1eb8ff9640 1 -- 172.21.15.96:0/749843086 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4003e20 con 0x55c4a5ce0620 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1eb8ff9640 1 -- 172.21.15.96:0/749843086 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4034040 con 0x55c4a5ce0620 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 -- 172.21.15.96:0/749843086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 msgr2=0x55c4a5cf7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.107 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf7290 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009bf0 tx=0x7f1ec402a950 comp rx=0 tx=0).stop 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 -- 172.21.15.96:0/749843086 shutdown_connections 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ceb800 0x55c4a5d03660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cdbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/749843086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf7290 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 -- 172.21.15.96:0/749843086 >> 172.21.15.96:0/749843086 conn(0x55c4a5cdcb30 msgr2=0x55c4a5cdb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 -- 172.21.15.96:0/749843086 shutdown_connections 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.102+0000 7f1edc1a7740 1 -- 172.21.15.96:0/749843086 wait complete. 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 -- start start 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 Processor -- start 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cf0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5d5fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41abff0 con 0x55c4a5ceb800 2024-04-25T21:00:52.108 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ad460 con 0x55c4a5ce0f40 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed41ac6c0 con 0x55c4a5ce0620 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1edc1a7740 10 client.0 _ll_register_callbacks cb 0x7f1ecc011520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5d5fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5d5fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39724/0 (socket says 172.21.15.96:39724) 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 -- 172.21.15.96:0/2274434468 learned_addr learned my addr 172.21.15.96:0/2274434468 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed27fc640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cf0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 -- 172.21.15.96:0/2274434468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 msgr2=0x55c4a5cf0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 -- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 msgr2=0x55c4a5cf0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cf0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 -- 172.21.15.96:0/2274434468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a5ce0d70 con 0x55c4a5ceb800 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1ed37fe640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5d5fed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55c4a5cddee0 tx=0x7f1ecc005e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3b64c641bd25f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:00:52.109 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1eb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/2274434468 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00fc00 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.103+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/2274434468 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f1ecc003790 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ec4003e20 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/2274434468 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ecc011020 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4a5d61350 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/2274434468 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ec4003e20 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/2274434468 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1ecc044050 con 0x55c4a5ceb800 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 20 client.15687 populate_metadata read hostname 'smithi096' 2024-04-25T21:00:52.110 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5d61aa0 0x55c4a5d63ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:00:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 4 client.15687 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:00:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4a5c4d200 con 0x55c4a5d61aa0 2024-04-25T21:00:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.104+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5d61aa0 0x55c4a5d63ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:00:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.105+0000 7f1eb6ff5640 10 client.15687 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:52.111 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.105+0000 7f1ed2ffd640 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5d61aa0 0x55c4a5d63ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54759a0697d1369c server_cookie=1e8331a7f455b383 in_seq=0 out_seq=0 2024-04-25T21:00:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1eb6ff5640 1 -- 172.21.15.96:0/2274434468 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4a5c4d200 con 0x55c4a5d61aa0 2024-04-25T21:00:52.149 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1eb6ff5640 10 client.15687 handle_command_reply: tid=0 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 client.15687 shutdown 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 msgr2=0x55c4a5d5fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5d5fed0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x55c4a5cddee0 tx=0x7f1ecc005e20 comp rx=0 tx=0).stop 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 shutdown_connections 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55c4a5d61aa0 0x55c4a5d63ec0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c4a5ceb800 0x55c4a5d5fed0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c4a5ce0f40 0x55c4a5cf0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 --2- 172.21.15.96:0/2274434468 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55c4a5ce0620 0x55c4a5cf0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 >> 172.21.15.96:0/2274434468 conn(0x55c4a5cdcb30 msgr2=0x55c4a5cdb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1eb6ff5640 0 client.15687 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 shutdown_connections 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 1 -- 172.21.15.96:0/2274434468 wait complete. 2024-04-25T21:00:52.150 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.148+0000 7f1edc1a7740 20 client.15687 trim_cache size 0 max 16384 2024-04-25T21:00:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:00:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:00:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:00:52.151 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1ebc0a3dc0 msgr2=0x7f1ebc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1ebc0a3dc0 0x7f1ebc0a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x55c4a5cddf50 tx=0x7f1ecc005f90 comp rx=0 tx=0).stop 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 msgr2=0x7f1ed4198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 0x7f1ed4198c00 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e9f0 tx=0x7f1ec800eeb0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 shutdown_connections 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f1ebc0a3dc0 0x7f1ebc0a6280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1ed410c5d0 0x7f1ed419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ed4103b20 0x7f1ed4198c00 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 --2- 172.21.15.96:0/555126969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ed4103200 0x7f1ed41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 >> 172.21.15.96:0/555126969 conn(0x7f1ed40771d0 msgr2=0x7f1ed40fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 shutdown_connections 2024-04-25T21:00:52.152 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:00:52.149+0000 7f1ed948a640 1 -- 172.21.15.96:0/555126969 wait complete. 2024-04-25T21:00:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:52 smithi097 ceph-mon[32667]: pgmap v100: 97 pgs: 97 active+clean; 38 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 371 op/s 2024-04-25T21:00:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:52 smithi139 ceph-mon[33027]: pgmap v100: 97 pgs: 97 active+clean; 38 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 371 op/s 2024-04-25T21:00:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:52 smithi096 ceph-mon[29288]: pgmap v100: 97 pgs: 97 active+clean; 38 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.7 MiB/s wr, 371 op/s 2024-04-25T21:00:52.843 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0xffcac 2024-04-25T21:00:52.948 INFO:tasks.workunit.client.0.smithi096.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-04-25T21:00:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:54 smithi097 ceph-mon[32667]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.7 MiB/s wr, 255 op/s 2024-04-25T21:00:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:54 smithi139 ceph-mon[33027]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.7 MiB/s wr, 255 op/s 2024-04-25T21:00:54.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:54 smithi096 ceph-mon[29288]: pgmap v101: 97 pgs: 97 active+clean; 38 MiB data, 579 MiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 1.7 MiB/s wr, 255 op/s 2024-04-25T21:00:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:56 smithi097 ceph-mon[32667]: pgmap v102: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 624 op/s 2024-04-25T21:00:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:56 smithi139 ceph-mon[33027]: pgmap v102: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 624 op/s 2024-04-25T21:00:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:56 smithi096 ceph-mon[29288]: pgmap v102: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.9 MiB/s wr, 624 op/s 2024-04-25T21:00:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:00:58 smithi097 ceph-mon[32667]: pgmap v103: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 531 op/s 2024-04-25T21:00:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:00:58 smithi139 ceph-mon[33027]: pgmap v103: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 531 op/s 2024-04-25T21:00:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:00:58 smithi096 ceph-mon[29288]: pgmap v103: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 531 op/s 2024-04-25T21:01:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:00 smithi097 ceph-mon[32667]: pgmap v104: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 531 op/s 2024-04-25T21:01:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:00 smithi139 ceph-mon[33027]: pgmap v104: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 531 op/s 2024-04-25T21:01:00.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:00 smithi096 ceph-mon[29288]: pgmap v104: 97 pgs: 97 active+clean; 38 MiB data, 631 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 531 op/s 2024-04-25T21:01:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:02 smithi139 ceph-mon[33027]: pgmap v105: 97 pgs: 97 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 840 op/s 2024-04-25T21:01:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:02 smithi096 ceph-mon[29288]: pgmap v105: 97 pgs: 97 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 840 op/s 2024-04-25T21:01:02.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:02 smithi097 ceph-mon[32667]: pgmap v105: 97 pgs: 97 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 5.1 MiB/s wr, 840 op/s 2024-04-25T21:01:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:04 smithi139 ceph-mon[33027]: pgmap v106: 97 pgs: 97 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 677 op/s 2024-04-25T21:01:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:04 smithi096 ceph-mon[29288]: pgmap v106: 97 pgs: 97 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 677 op/s 2024-04-25T21:01:04.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:04 smithi097 ceph-mon[32667]: pgmap v106: 97 pgs: 97 active+clean; 41 MiB data, 645 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 677 op/s 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 -- 172.21.15.96:0/1292560145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec101aa0 msgr2=0x7f27ec105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/1292560145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec101aa0 0x7f27ec105a90 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e002a410 comp rx=0 tx=0).stop 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 -- 172.21.15.96:0/1292560145 shutdown_connections 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/1292560145 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 0x7f27ec10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/1292560145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec101aa0 0x7f27ec105a90 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/1292560145 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec1010f0 0x7f27ec1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 -- 172.21.15.96:0/1292560145 >> 172.21.15.96:0/1292560145 conn(0x7f27ec077fd0 msgr2=0x7f27ec0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 -- 172.21.15.96:0/1292560145 shutdown_connections 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.240+0000 7f27f0cd0640 1 -- 172.21.15.96:0/1292560145 wait complete. 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 Processor -- start 2024-04-25T21:01:05.244 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 -- start start 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 0x7f27ec10e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 0x7f27ec10ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec10f150 0x7f27ec1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1121f0 con 0x7f27ec1010f0 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec112070 con 0x7f27ec106170 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27f0cd0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec112370 con 0x7f27ec10f150 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 0x7f27ec10e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 0x7f27ec10e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33190/0 (socket says 172.21.15.96:33190) 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27ea59c640 1 -- 172.21.15.96:0/2760701761 learned_addr learned my addr 172.21.15.96:0/2760701761 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 0x7f27ec10ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.241+0000 7f27ead9d640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec10f150 0x7f27ec1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27ea59c640 1 -- 172.21.15.96:0/2760701761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec10f150 msgr2=0x7f27ec1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.245 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27ea59c640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec10f150 0x7f27ec1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27ea59c640 1 -- 172.21.15.96:0/2760701761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 msgr2=0x7f27ec10ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27ea59c640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 0x7f27ec10ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27ea59c640 1 -- 172.21.15.96:0/2760701761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27ec1a6fa0 con 0x7f27ec1010f0 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 0x7f27ec10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27ea59c640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 0x7f27ec10e6d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00e9f0 tx=0x7f27dc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27db7fe640 1 -- 172.21.15.96:0/2760701761 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00cd80 con 0x7f27ec1010f0 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27ec1a7290 con 0x7f27ec1010f0 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27db7fe640 1 -- 172.21.15.96:0/2760701761 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27dc0054d0 con 0x7f27ec1010f0 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27db7fe640 1 -- 172.21.15.96:0/2760701761 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc010690 con 0x7f27ec1010f0 2024-04-25T21:01:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.242+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27ec1a77b0 con 0x7f27ec1010f0 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.244+0000 7f27db7fe640 1 -- 172.21.15.96:0/2760701761 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f27dc004e90 con 0x7f27ec1010f0 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 Processor -- start 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 -- start start 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27db7fe640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f27d40a3dc0 0x7f27d40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06620 0x556950b06a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 0x556950b112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556950a6b970 con 0x556950b06620 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556950a084a0 con 0x556950b06f40 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27f39e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556950a67930 con 0x556950b11800 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ead9d640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f27d40a3dc0 0x7f27d40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27db7fe640 1 -- 172.21.15.96:0/2760701761 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f27dc0d0b90 con 0x7f27ec1010f0 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 0x556950b112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ea59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06620 0x556950b06a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ead9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 0x556950b112c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:33798/0 (socket says 172.21.15.96:33798) 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27e9d9b640 1 -- 172.21.15.96:0/2709216036 learned_addr learned my addr 172.21.15.96:0/2709216036 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ea59c640 1 -- 172.21.15.96:0/2709216036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 msgr2=0x556950b13bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ea59c640 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b13bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ea59c640 1 -- 172.21.15.96:0/2709216036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 msgr2=0x556950b112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ea59c640 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 0x556950b112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27ea59c640 1 -- 172.21.15.96:0/2709216036 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55695077e210 con 0x556950b06620 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.245+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 0x556950b112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27ead9d640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f27d40a3dc0 0x7f27d40a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009fd0 tx=0x7f27e4009d80 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27ea59c640 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06620 0x556950b06a00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0ce350 tx=0x7f27dc09c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b104b513940814e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27d8ff9640 1 -- 172.21.15.96:0/2709216036 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc09d2a0 con 0x556950b06620 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27d8ff9640 1 -- 172.21.15.96:0/2709216036 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27dc09cba0 con 0x556950b06620 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27d8ff9640 1 -- 172.21.15.96:0/2709216036 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0ce5d0 con 0x556950b06620 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27f39e2740 1 -- 172.21.15.96:0/2709216036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06620 msgr2=0x556950b06a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.246+0000 7f27f39e2740 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06620 0x556950b06a00 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0ce350 tx=0x7f27dc09c7a0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- 172.21.15.96:0/2709216036 shutdown_connections 2024-04-25T21:01:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06f40 0x556950b112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 --2- 172.21.15.96:0/2709216036 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06620 0x556950b06a00 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- 172.21.15.96:0/2709216036 >> 172.21.15.96:0/2709216036 conn(0x556950b02b30 msgr2=0x556950b02f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- 172.21.15.96:0/2709216036 shutdown_connections 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- 172.21.15.96:0/2709216036 wait complete. 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- start start 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 Processor -- start 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b11160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b0e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b0e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556950b14430 con 0x556950b06f40 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x556950a060b0 con 0x556950b06620 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x556950b145b0 con 0x556950b11800 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27f39e2740 10 client.0 _ll_register_callbacks cb 0x5569509f3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b0e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b0e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33208/0 (socket says 172.21.15.96:33208) 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27e9d9b640 1 -- 172.21.15.96:0/1622014707 learned_addr learned my addr 172.21.15.96:0/1622014707 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.247+0000 7f27ead9d640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b0e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27ea59c640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b11160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27e9d9b640 1 -- 172.21.15.96:0/1622014707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 msgr2=0x556950b0e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b0e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27e9d9b640 1 -- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 msgr2=0x556950b11160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b11160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27e9d9b640 1 -- 172.21.15.96:0/1622014707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55695084a620 con 0x556950b06f40 2024-04-25T21:01:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27ea59c640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b11160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b0e450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009230 tx=0x7f27e00316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876c19fa768d085d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27d1ffb640 1 -- 172.21.15.96:0/1622014707 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0035040 con 0x556950b06f40 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27d1ffb640 1 -- 172.21.15.96:0/1622014707 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f27e0004240 con 0x556950b06f40 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27dc09cba0 con 0x556950b06f40 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27d1ffb640 1 -- 172.21.15.96:0/1622014707 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27e0004440 con 0x556950b06f40 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27f39e2740 10 client.15699 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556950b20360 con 0x556950b06f40 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27d1ffb640 1 -- 172.21.15.96:0/1622014707 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27dc09cba0 con 0x556950b06f40 2024-04-25T21:01:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27d1ffb640 1 -- 172.21.15.96:0/1622014707 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f27e004e050 con 0x556950b06f40 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.248+0000 7f27f39e2740 10 client.15699 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27f39e2740 10 client.15699 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27f39e2740 20 client.15699 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27f39e2740 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b21820 0x556950b23ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27f39e2740 4 client.15699 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556950a79390 con 0x556950b21820 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27ea59c640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b21820 0x556950b23ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27ea59c640 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b21820 0x556950b23ca0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c037567d61444cc9 server_cookie=913fbd7d9a65dc1 in_seq=0 out_seq=0 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.249+0000 7f27d1ffb640 10 client.15699 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27d1ffb640 1 -- 172.21.15.96:0/1622014707 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556950a79390 con 0x556950b21820 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27d1ffb640 10 client.15699 handle_command_reply: tid=0 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 client.15699 shutdown 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 msgr2=0x556950b0e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.259 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b0e450 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009230 tx=0x7f27e00316d0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 shutdown_connections 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b21820 0x556950b23ca0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b0e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b0e450 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27d1ffb640 0 client.15699 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 --2- 172.21.15.96:0/1622014707 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b11160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 >> 172.21.15.96:0/1622014707 conn(0x556950b02b30 msgr2=0x556950b088b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 shutdown_connections 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 1 -- 172.21.15.96:0/1622014707 wait complete. 2024-04-25T21:01:05.260 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.250+0000 7f27f39e2740 20 client.15699 trim_cache size 0 max 16384 2024-04-25T21:01:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.264+0000 7f27f39e2740 1 Processor -- start 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 -- start start 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b1d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b01b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b29570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1b2520 con 0x556950b06f40 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1afd90 con 0x556950b06620 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27f39e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1adc40 con 0x556950b11800 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b01b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b01b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33212/0 (socket says 172.21.15.96:33212) 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 -- 172.21.15.96:0/4195619516 learned_addr learned my addr 172.21.15.96:0/4195619516 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27ea59c640 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b1d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27ead9d640 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b29570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 -- 172.21.15.96:0/4195619516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 msgr2=0x556950b29570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b29570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 -- 172.21.15.96:0/4195619516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 msgr2=0x556950b1d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b1d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 -- 172.21.15.96:0/4195619516 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556950b13bf0 con 0x556950b06f40 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b01b30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f27e002a930 tx=0x7f27e002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3180a7fb258722 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27d3fff640 1 -- 172.21.15.96:0/4195619516 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e0035030 con 0x556950b06f40 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27d3fff640 1 -- 172.21.15.96:0/4195619516 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0004970 con 0x556950b06f40 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.265+0000 7f27d3fff640 1 -- 172.21.15.96:0/4195619516 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e00467e0 con 0x556950b06f40 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 -- 172.21.15.96:0/4195619516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 msgr2=0x556950b01b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b01b30 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f27e002a930 tx=0x7f27e002a410 comp rx=0 tx=0).stop 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 -- 172.21.15.96:0/4195619516 shutdown_connections 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b29570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b01b30 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 --2- 172.21.15.96:0/4195619516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b1d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 -- 172.21.15.96:0/4195619516 >> 172.21.15.96:0/4195619516 conn(0x556950b02b30 msgr2=0x556950b01420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 -- 172.21.15.96:0/4195619516 shutdown_connections 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 -- 172.21.15.96:0/4195619516 wait complete. 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.266+0000 7f27f39e2740 1 -- start start 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 Processor -- start 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b19c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b16fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b17530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1ae290 con 0x556950b06f40 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1af6c0 con 0x556950b06620 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f27ec1ae920 con 0x556950b11800 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 10 client.0 _ll_register_callbacks cb 0x7f27e0005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b16fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b16fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:33220/0 (socket says 172.21.15.96:33220) 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 -- 172.21.15.96:0/370501227 learned_addr learned my addr 172.21.15.96:0/370501227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27ea59c640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b19c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27ead9d640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b17530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 -- 172.21.15.96:0/370501227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 msgr2=0x556950b17530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b17530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 -- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 msgr2=0x556950b19c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b19c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 -- 172.21.15.96:0/370501227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556950b17d60 con 0x556950b06f40 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27ead9d640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27e9d9b640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b16fd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e00052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65b57465412c410 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27d1ffb640 1 -- 172.21.15.96:0/370501227 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27e00316b0 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27d1ffb640 1 -- 172.21.15.96:0/370501227 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f27e0046470 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.267+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27e0004970 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27d1ffb640 1 -- 172.21.15.96:0/370501227 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f27e0068050 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556950b87e90 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27d1ffb640 1 -- 172.21.15.96:0/370501227 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27e0004970 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27d1ffb640 1 -- 172.21.15.96:0/370501227 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f27e004e040 con 0x556950b06f40 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 20 client.15705 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b885d0 0x556950b8a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 4 client.15705 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556950a73200 con 0x556950b885d0 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.268+0000 7f27ea59c640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b885d0 0x556950b8a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.269+0000 7f27d1ffb640 10 client.15705 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.269+0000 7f27ea59c640 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b885d0 0x556950b8a9f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=634c0ac89d81658e server_cookie=ad699dc92c48362b in_seq=0 out_seq=0 2024-04-25T21:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27d1ffb640 1 -- 172.21.15.96:0/370501227 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556950a73200 con 0x556950b885d0 2024-04-25T21:01:05.285 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27d1ffb640 10 client.15705 handle_command_reply: tid=0 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 client.15705 shutdown 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 msgr2=0x556950b16fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b16fd0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f27e0009900 tx=0x7f27e00052f0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 shutdown_connections 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x556950b885d0 0x556950b8a9f0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x556950b11800 0x556950b17530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556950b06f40 0x556950b16fd0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 --2- 172.21.15.96:0/370501227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556950b06620 0x556950b19c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.286 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27d1ffb640 0 client.15705 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 >> 172.21.15.96:0/370501227 conn(0x556950b02b30 msgr2=0x556950b01cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 shutdown_connections 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 1 -- 172.21.15.96:0/370501227 wait complete. 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.284+0000 7f27f39e2740 20 client.15705 trim_cache size 0 max 16384 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:01:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f27d40a3dc0 msgr2=0x7f27d40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f27d40a3dc0 0x7f27d40a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27e4009fd0 tx=0x7f27e4009d80 comp rx=0 tx=0).stop 2024-04-25T21:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 msgr2=0x7f27ec10e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 0x7f27ec10e6d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f27dc00e9f0 tx=0x7f27dc00edf0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 shutdown_connections 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f27d40a3dc0 0x7f27d40a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f27ec10f150 0x7f27ec1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27ec106170 0x7f27ec10ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 --2- 172.21.15.96:0/2760701761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27ec1010f0 0x7f27ec10e6d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 >> 172.21.15.96:0/2760701761 conn(0x7f27ec077fd0 msgr2=0x7f27ec0fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 shutdown_connections 2024-04-25T21:01:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:05.285+0000 7f27f0cd0640 1 -- 172.21.15.96:0/2760701761 wait complete. 2024-04-25T21:01:06.593 INFO:tasks.workunit.client.0.smithi096.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-04-25T21:01:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:06 smithi139 ceph-mon[33027]: pgmap v107: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 972 op/s 2024-04-25T21:01:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:06 smithi096 ceph-mon[29288]: pgmap v107: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 972 op/s 2024-04-25T21:01:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:06 smithi097 ceph-mon[32667]: pgmap v107: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.9 MiB/s wr, 972 op/s 2024-04-25T21:01:06.929 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xffcdd 2024-04-25T21:01:08.544 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xffd97 2024-04-25T21:01:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:08 smithi139 ceph-mon[33027]: pgmap v108: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 603 op/s 2024-04-25T21:01:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:08 smithi096 ceph-mon[29288]: pgmap v108: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 603 op/s 2024-04-25T21:01:08.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:08 smithi097 ceph-mon[32667]: pgmap v108: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 603 op/s 2024-04-25T21:01:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:10 smithi096 ceph-mon[29288]: pgmap v109: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 603 op/s 2024-04-25T21:01:10.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:10 smithi097 ceph-mon[32667]: pgmap v109: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 603 op/s 2024-04-25T21:01:10.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:10 smithi139 ceph-mon[33027]: pgmap v109: 97 pgs: 97 active+clean; 45 MiB data, 676 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.7 MiB/s wr, 603 op/s 2024-04-25T21:01:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:12 smithi096 ceph-mon[29288]: pgmap v110: 97 pgs: 97 active+clean; 36 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 861 op/s 2024-04-25T21:01:12.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:12 smithi097 ceph-mon[32667]: pgmap v110: 97 pgs: 97 active+clean; 36 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 861 op/s 2024-04-25T21:01:12.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:12 smithi139 ceph-mon[33027]: pgmap v110: 97 pgs: 97 active+clean; 36 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 5.5 MiB/s wr, 861 op/s 2024-04-25T21:01:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:14 smithi096 ceph-mon[29288]: pgmap v111: 97 pgs: 97 active+clean; 36 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 552 op/s 2024-04-25T21:01:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:14 smithi097 ceph-mon[32667]: pgmap v111: 97 pgs: 97 active+clean; 36 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 552 op/s 2024-04-25T21:01:14.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:14 smithi139 ceph-mon[33027]: pgmap v111: 97 pgs: 97 active+clean; 36 MiB data, 723 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 3.6 MiB/s wr, 552 op/s 2024-04-25T21:01:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:16 smithi096 ceph-mon[29288]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 770 op/s 2024-04-25T21:01:16.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:16 smithi097 ceph-mon[32667]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 770 op/s 2024-04-25T21:01:16.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:16 smithi139 ceph-mon[33027]: pgmap v112: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.1 MiB/s wr, 770 op/s 2024-04-25T21:01:17.066 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0xffdb8 2024-04-25T21:01:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2121922544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4101aa0 msgr2=0x7fa4a4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2121922544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4101aa0 0x7fa4a4105a90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa494009900 tx=0x7fa49402a410 comp rx=0 tx=0).stop 2024-04-25T21:01:18.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2121922544 shutdown_connections 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2121922544 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4106170 0x7fa4a410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2121922544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4101aa0 0x7fa4a4105a90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2121922544 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 0x7fa4a41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.375+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2121922544 >> 172.21.15.96:0/2121922544 conn(0x7fa4a4077fd0 msgr2=0x7fa4a40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2121922544 shutdown_connections 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2121922544 wait complete. 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 Processor -- start 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 -- start start 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 0x7fa4a41973a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 0x7fa4a41978e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4197e20 0x7fa4a41a6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41110b0 con 0x7fa4a4106170 2024-04-25T21:01:18.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4110f30 con 0x7fa4a41010f0 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4ab62a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a4111230 con 0x7fa4a4197e20 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4a8bc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 0x7fa4a41978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4a8bc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 0x7fa4a41978e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45490/0 (socket says 172.21.15.96:45490) 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4a8bc5640 1 -- 172.21.15.96:0/2192181217 learned_addr learned my addr 172.21.15.96:0/2192181217 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4197e20 0x7fa4a41a6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.376+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 0x7fa4a41973a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a8bc5640 1 -- 172.21.15.96:0/2192181217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4197e20 msgr2=0x7fa4a41a6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4197e20 0x7fa4a41a6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a8bc5640 1 -- 172.21.15.96:0/2192181217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 msgr2=0x7fa4a41973a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 0x7fa4a41973a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a8bc5640 1 -- 172.21.15.96:0/2192181217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4a41a6fe0 con 0x7fa4a4106170 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 0x7fa4a41973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:18.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 0x7fa4a41978e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa494031a20 tx=0x7fa494031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa49a7fc640 1 -- 172.21.15.96:0/2192181217 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49403f030 con 0x7fa4a4106170 2024-04-25T21:01:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa49a7fc640 1 -- 172.21.15.96:0/2192181217 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4940044c0 con 0x7fa4a4106170 2024-04-25T21:01:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4a41a7270 con 0x7fa4a4106170 2024-04-25T21:01:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa49a7fc640 1 -- 172.21.15.96:0/2192181217 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494035070 con 0x7fa4a4106170 2024-04-25T21:01:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.377+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4a41a7760 con 0x7fa4a4106170 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.379+0000 7fa49a7fc640 1 -- 172.21.15.96:0/2192181217 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa494042030 con 0x7fa4a4106170 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 Processor -- start 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 -- start start 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa49a7fc640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa4900a3dc0 0x7fa4900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1d94a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94f40 0x564ed1d9f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d9f800 0x564ed1da1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ed1cf9970 con 0x564ed1d94f40 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ed1c964a0 con 0x564ed1d9f800 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4ae419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564ed1cf5930 con 0x564ed1d94620 2024-04-25T21:01:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa49a7fc640 1 -- 172.21.15.96:0/2192181217 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fa4940f0ab0 con 0x7fa4a4106170 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa4900a3dc0 0x7fa4900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4a9bc7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d9f800 0x564ed1da1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4a93c6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1d94a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4a8bc5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94f40 0x564ed1d9f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4a9bc7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d9f800 0x564ed1da1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:37428/0 (socket says 172.21.15.96:37428) 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.380+0000 7fa4a9bc7640 1 -- 172.21.15.96:0/3884611295 learned_addr learned my addr 172.21.15.96:0/3884611295 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a93c6640 1 -- 172.21.15.96:0/3884611295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d9f800 msgr2=0x564ed1da1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d9f800 0x564ed1da1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a93c6640 1 -- 172.21.15.96:0/3884611295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94f40 msgr2=0x564ed1d9f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94f40 0x564ed1d9f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a93c6640 1 -- 172.21.15.96:0/3884611295 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed1a0c210 con 0x564ed1d94620 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94f40 0x564ed1d9f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa4900a3dc0 0x7fa4900a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009fd0 tx=0x7fa49c009f40 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.381+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1d94a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fd0 tx=0x7fa4a00118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8334f15204ce522 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.385+0000 7fa48f7fe640 1 -- 172.21.15.96:0/3884611295 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0004030 con 0x564ed1d94620 2024-04-25T21:01:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.385+0000 7fa48f7fe640 1 -- 172.21.15.96:0/3884611295 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a00054d0 con 0x564ed1d94620 2024-04-25T21:01:18.386 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.385+0000 7fa48f7fe640 1 -- 172.21.15.96:0/3884611295 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0004ce0 con 0x564ed1d94620 2024-04-25T21:01:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 -- 172.21.15.96:0/3884611295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 msgr2=0x564ed1d94a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1d94a00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009fd0 tx=0x7fa4a00118d0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 -- 172.21.15.96:0/3884611295 shutdown_connections 2024-04-25T21:01:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d9f800 0x564ed1da1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94f40 0x564ed1d9f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3884611295 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1d94a00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 -- 172.21.15.96:0/3884611295 >> 172.21.15.96:0/3884611295 conn(0x564ed1d90b30 msgr2=0x564ed1d90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 -- 172.21.15.96:0/3884611295 shutdown_connections 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.386+0000 7fa4ae419740 1 -- 172.21.15.96:0/3884611295 wait complete. 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 -- start start 2024-04-25T21:01:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 Processor -- start 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94620 0x564ed1d9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94f40 0x564ed1d9c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d9c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564ed1da2430 con 0x564ed1d9f800 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x564ed1c940b0 con 0x564ed1d94620 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x564ed1da25b0 con 0x564ed1d94f40 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.387+0000 7fa4ae419740 10 client.0 _ll_register_callbacks cb 0x564ed1c81d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a9bc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d9c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a8bc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94f40 0x564ed1d9c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94620 0x564ed1d9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a8bc5640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94f40 0x564ed1d9c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:41330/0 (socket says 172.21.15.96:41330) 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a9bc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d9c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45508/0 (socket says 172.21.15.96:45508) 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a8bc5640 1 -- 172.21.15.96:0/3398319338 learned_addr learned my addr 172.21.15.96:0/3398319338 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 -- 172.21.15.96:0/3398319338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94f40 msgr2=0x564ed1d9c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94f40 0x564ed1d9c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 -- 172.21.15.96:0/3398319338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 msgr2=0x564ed1d9c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d9c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 -- 172.21.15.96:0/3398319338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed1ad8620 con 0x564ed1d94620 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d9c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94620 0x564ed1d9f180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0012fd0 tx=0x7fa4a00050c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed7cde5f8be6ebd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa48cff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa48cff9640 1 -- 172.21.15.96:0/3398319338 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0029020 con 0x564ed1d94620 2024-04-25T21:01:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa48cff9640 1 -- 172.21.15.96:0/3398319338 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fa4a000f540 con 0x564ed1d94620 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.388+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a00054d0 con 0x564ed1d94620 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa48cff9640 1 -- 172.21.15.96:0/3398319338 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa4a000f780 con 0x564ed1d94620 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 10 client.25190 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ed1dae380 con 0x564ed1d94620 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa48cff9640 1 -- 172.21.15.96:0/3398319338 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a00054d0 con 0x564ed1d94620 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa48cff9640 1 -- 172.21.15.96:0/3398319338 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa4a0025920 con 0x564ed1d94620 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 10 client.25190 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 10 client.25190 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 20 client.25190 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1daf7e0 0x564ed1db1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 4 client.25190 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed1d07390 con 0x564ed1daf7e0 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1daf7e0 0x564ed1db1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa48cff9640 10 client.25190 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.389+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1daf7e0 0x564ed1db1bd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17da1ff4036772a6 server_cookie=d1ce55fcbba8b9d5 in_seq=0 out_seq=0 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa48cff9640 1 -- 172.21.15.96:0/3398319338 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ed1d07390 con 0x564ed1daf7e0 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa48cff9640 10 client.25190 handle_command_reply: tid=0 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 client.25190 shutdown 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94620 msgr2=0x564ed1d9f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94620 0x564ed1d9f180 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0012fd0 tx=0x7fa4a00050c0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 shutdown_connections 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1daf7e0 0x564ed1db1bd0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d9c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa48cff9640 0 client.25190 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94f40 0x564ed1d9c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 --2- 172.21.15.96:0/3398319338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94620 0x564ed1d9f180 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 >> 172.21.15.96:0/3398319338 conn(0x564ed1d90b30 msgr2=0x564ed1d968b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:18.395 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 shutdown_connections 2024-04-25T21:01:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 1 -- 172.21.15.96:0/3398319338 wait complete. 2024-04-25T21:01:18.396 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.390+0000 7fa4ae419740 20 client.25190 trim_cache size 0 max 16384 2024-04-25T21:01:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 Processor -- start 2024-04-25T21:01:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 -- start start 2024-04-25T21:01:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 0x564ed1dab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d9ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d9f800 0x564ed1d8fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41b2500 con 0x564ed1d94620 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41afd70 con 0x564ed1d94f40 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4ae419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41adbc0 con 0x564ed1d9f800 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 0x564ed1dab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 0x564ed1dab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45524/0 (socket says 172.21.15.96:45524) 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 -- 172.21.15.96:0/619937159 learned_addr learned my addr 172.21.15.96:0/619937159 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d9ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d9f800 0x564ed1d8fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 -- 172.21.15.96:0/619937159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d9f800 msgr2=0x564ed1d8fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d9f800 0x564ed1d8fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 -- 172.21.15.96:0/619937159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 msgr2=0x564ed1d9ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d9ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 -- 172.21.15.96:0/619937159 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed1d9b460 con 0x564ed1d94620 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.405+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 0x564ed1dab2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009ef0 tx=0x7fa4a0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85480e8ee64974f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa48effd640 1 -- 172.21.15.96:0/619937159 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a003d070 con 0x564ed1d94620 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa48effd640 1 -- 172.21.15.96:0/619937159 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a0019450 con 0x564ed1d94620 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa48effd640 1 -- 172.21.15.96:0/619937159 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4a0029040 con 0x564ed1d94620 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 -- 172.21.15.96:0/619937159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 msgr2=0x564ed1dab2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 0x564ed1dab2f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fa4a0009ef0 tx=0x7fa4a0005970 comp rx=0 tx=0).stop 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 -- 172.21.15.96:0/619937159 shutdown_connections 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d9f800 0x564ed1d8fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d9ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 --2- 172.21.15.96:0/619937159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d94620 0x564ed1dab2f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 -- 172.21.15.96:0/619937159 >> 172.21.15.96:0/619937159 conn(0x564ed1d90b30 msgr2=0x564ed1d8f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 -- 172.21.15.96:0/619937159 shutdown_connections 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.406+0000 7fa4ae419740 1 -- 172.21.15.96:0/619937159 wait complete. 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 -- start start 2024-04-25T21:01:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 Processor -- start 2024-04-25T21:01:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1da79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d98d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d99260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41ae230 con 0x564ed1d9f800 2024-04-25T21:01:18.412 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41af6a0 con 0x564ed1d94f40 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fa4a41ae900 con 0x564ed1d94620 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4ae419740 10 client.0 _ll_register_callbacks cb 0x7fa4a0009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4a9bc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d99260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4a9bc7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d99260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:45530/0 (socket says 172.21.15.96:45530) 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4a9bc7640 1 -- 172.21.15.96:0/1082426739 learned_addr learned my addr 172.21.15.96:0/1082426739 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1da79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.407+0000 7fa4a8bc5640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d98d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4a9bc7640 1 -- 172.21.15.96:0/1082426739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 msgr2=0x564ed1da79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1da79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4a9bc7640 1 -- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 msgr2=0x564ed1d98d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d98d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4a9bc7640 1 -- 172.21.15.96:0/1082426739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed1d9cd60 con 0x564ed1d9f800 2024-04-25T21:01:18.413 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4a9bc7640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d99260 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00fc20 tx=0x7fa49c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f15ca3ba2dc01a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa48cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa48cff9640 1 -- 172.21.15.96:0/1082426739 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa49c02e070 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa48cff9640 1 -- 172.21.15.96:0/1082426739 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fa49c004b20 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4a0019450 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa48cff9640 1 -- 172.21.15.96:0/1082426739 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa49c026040 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ed1e15720 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa48cff9640 1 -- 172.21.15.96:0/1082426739 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4a0019450 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa48cff9640 1 -- 172.21.15.96:0/1082426739 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fa49c031030 con 0x564ed1d9f800 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 20 client.15723 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:18.414 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1e15f30 0x564ed1e18350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 4 client.15723 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.408+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564ed1d01200 con 0x564ed1e15f30 2024-04-25T21:01:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.409+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1e15f30 0x564ed1e18350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.409+0000 7fa4a93c6640 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1e15f30 0x564ed1e18350 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294e9c6eaac6711c server_cookie=93a610802874a7fe in_seq=0 out_seq=0 2024-04-25T21:01:18.415 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.409+0000 7fa48cff9640 10 client.15723 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa48cff9640 1 -- 172.21.15.96:0/1082426739 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564ed1d01200 con 0x564ed1e15f30 2024-04-25T21:01:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa48cff9640 10 client.15723 handle_command_reply: tid=0 2024-04-25T21:01:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 client.15723 shutdown 2024-04-25T21:01:18.432 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 msgr2=0x564ed1d99260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d99260 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa49c00fc20 tx=0x7fa49c009500 comp rx=0 tx=0).stop 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 shutdown_connections 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x564ed1e15f30 0x564ed1e18350 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564ed1d9f800 0x564ed1d99260 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x564ed1d94f40 0x564ed1d98d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa48cff9640 0 client.15723 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 --2- 172.21.15.96:0/1082426739 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x564ed1d94620 0x564ed1da79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 >> 172.21.15.96:0/1082426739 conn(0x564ed1d90b30 msgr2=0x564ed1d979b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 shutdown_connections 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 1 -- 172.21.15.96:0/1082426739 wait complete. 2024-04-25T21:01:18.433 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.431+0000 7fa4ae419740 20 client.15723 trim_cache size 0 max 16384 2024-04-25T21:01:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:01:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:01:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:01:18.434 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa4900a3dc0 msgr2=0x7fa4900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa4900a3dc0 0x7fa4900a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa49c009fd0 tx=0x7fa49c009f40 comp rx=0 tx=0).stop 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 msgr2=0x7fa4a41978e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 0x7fa4a41978e0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa494031a20 tx=0x7fa494031a50 comp rx=0 tx=0).stop 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 shutdown_connections 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fa4900a3dc0 0x7fa4900a6280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fa4a4197e20 0x7fa4a41a6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.435 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4a4106170 0x7fa4a41978e0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 --2- 172.21.15.96:0/2192181217 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa4a41010f0 0x7fa4a41973a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 >> 172.21.15.96:0/2192181217 conn(0x7fa4a4077fd0 msgr2=0x7fa4a40ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 shutdown_connections 2024-04-25T21:01:18.436 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:18.432+0000 7fa4ab62a640 1 -- 172.21.15.96:0/2192181217 wait complete. 2024-04-25T21:01:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:18 smithi096 ceph-mon[29288]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-04-25T21:01:18.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:18 smithi097 ceph-mon[32667]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-04-25T21:01:18.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:18 smithi139 ceph-mon[33027]: pgmap v113: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-04-25T21:01:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:20 smithi096 ceph-mon[29288]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-04-25T21:01:20.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:20 smithi097 ceph-mon[32667]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-04-25T21:01:20.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:20 smithi139 ceph-mon[33027]: pgmap v114: 97 pgs: 97 active+clean; 40 MiB data, 731 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.2 MiB/s wr, 476 op/s 2024-04-25T21:01:22.666 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:22 smithi097 ceph-mon[32667]: pgmap v115: 97 pgs: 97 active+clean; 37 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 683 op/s 2024-04-25T21:01:22.673 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:22 smithi139 ceph-mon[33027]: pgmap v115: 97 pgs: 97 active+clean; 37 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 683 op/s 2024-04-25T21:01:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:22 smithi096 ceph-mon[29288]: pgmap v115: 97 pgs: 97 active+clean; 37 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.7 MiB/s wr, 683 op/s 2024-04-25T21:01:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:24 smithi096 ceph-mon[29288]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 425 op/s 2024-04-25T21:01:24.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:24 smithi097 ceph-mon[32667]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 425 op/s 2024-04-25T21:01:24.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:24 smithi139 ceph-mon[33027]: pgmap v116: 97 pgs: 97 active+clean; 37 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 425 op/s 2024-04-25T21:01:26.047 INFO:tasks.workunit.client.0.smithi096.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-04-25T21:01:26.047 INFO:tasks.workunit.client.0.smithi096.stdout:All 50000 operations completed A-OK! 2024-04-25T21:01:26.048 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-04-25T21:01:26.048 INFO:tasks.workunit.client.0.smithi096.stdout:Seed set to 1 2024-04-25T21:01:26.169 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T21:01:26.169 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T21:01:26.172 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T21:01:26.172 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T21:01:26.172 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T21:01:26.175 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero length punch hole 2024-04-25T21:01:26.178 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x4310e9 2024-04-25T21:01:26.184 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x954a45 2024-04-25T21:01:26.212 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x9bc582 2024-04-25T21:01:26.232 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x9eecbc 2024-04-25T21:01:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:26 smithi096 ceph-mon[29288]: pgmap v117: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 612 op/s 2024-04-25T21:01:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:26 smithi097 ceph-mon[32667]: pgmap v117: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 612 op/s 2024-04-25T21:01:26.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:26 smithi139 ceph-mon[33027]: pgmap v117: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.3 MiB/s wr, 612 op/s 2024-04-25T21:01:28.075 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x9fdd84 2024-04-25T21:01:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:28 smithi096 ceph-mon[29288]: pgmap v118: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 393 op/s 2024-04-25T21:01:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:28 smithi097 ceph-mon[32667]: pgmap v118: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 393 op/s 2024-04-25T21:01:28.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:28 smithi139 ceph-mon[33027]: pgmap v118: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 393 op/s 2024-04-25T21:01:30.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:30 smithi096 ceph-mon[29288]: pgmap v119: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 393 op/s 2024-04-25T21:01:30.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:30 smithi097 ceph-mon[32667]: pgmap v119: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 393 op/s 2024-04-25T21:01:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:30 smithi139 ceph-mon[33027]: pgmap v119: 97 pgs: 97 active+clean; 42 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.8 MiB/s wr, 393 op/s 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 -- 172.21.15.96:0/1551415680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc190102b80 msgr2=0x7fc190106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 --2- 172.21.15.96:0/1551415680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc190102b80 0x7fc190106bc0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc180009900 tx=0x7fc18002a410 comp rx=0 tx=0).stop 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 -- 172.21.15.96:0/1551415680 shutdown_connections 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 --2- 172.21.15.96:0/1551415680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901072a0 0x7fc19010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 --2- 172.21.15.96:0/1551415680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc190102b80 0x7fc190106bc0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 --2- 172.21.15.96:0/1551415680 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc1901021d0 0x7fc1901025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 -- 172.21.15.96:0/1551415680 >> 172.21.15.96:0/1551415680 conn(0x7fc1900793e0 msgr2=0x7fc1900797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 -- 172.21.15.96:0/1551415680 shutdown_connections 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 -- 172.21.15.96:0/1551415680 wait complete. 2024-04-25T21:01:31.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.527+0000 7fc196a83640 1 Processor -- start 2024-04-25T21:01:31.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 -- start start 2024-04-25T21:01:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901021d0 0x7fc1901986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1901072a0 0x7fc190198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19019cfa0 0x7fc19019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901121e0 con 0x7fc1901072a0 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc190112060 con 0x7fc1901021d0 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc196a83640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc190112360 con 0x7fc19019cfa0 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1901072a0 0x7fc190198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1901072a0 0x7fc190198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38240/0 (socket says 172.21.15.96:38240) 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc187fff640 1 -- 172.21.15.96:0/978370603 learned_addr learned my addr 172.21.15.96:0/978370603 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc19481f640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901021d0 0x7fc1901986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc195020640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19019cfa0 0x7fc19019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc19481f640 1 -- 172.21.15.96:0/978370603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19019cfa0 msgr2=0x7fc19019d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc19481f640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19019cfa0 0x7fc19019d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc19481f640 1 -- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1901072a0 msgr2=0x7fc190198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc19481f640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1901072a0 0x7fc190198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.528+0000 7fc19481f640 1 -- 172.21.15.96:0/978370603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc19019da90 con 0x7fc1901021d0 2024-04-25T21:01:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.529+0000 7fc19481f640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901021d0 0x7fc1901986e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.529+0000 7fc185ffb640 1 -- 172.21.15.96:0/978370603 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00ccc0 con 0x7fc1901021d0 2024-04-25T21:01:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.529+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc19019dd80 con 0x7fc1901021d0 2024-04-25T21:01:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.529+0000 7fc185ffb640 1 -- 172.21.15.96:0/978370603 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c005020 con 0x7fc1901021d0 2024-04-25T21:01:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.529+0000 7fc185ffb640 1 -- 172.21.15.96:0/978370603 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c004a00 con 0x7fc1901021d0 2024-04-25T21:01:31.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.529+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1901a5520 con 0x7fc1901021d0 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.531+0000 7fc19979d740 1 Processor -- start 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc185ffb640 1 -- 172.21.15.96:0/978370603 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc18c004ba0 con 0x7fc1901021d0 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 -- start start 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765caa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765caf40 0x5583765d52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765d5800 0x5583765d7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55837652f970 con 0x5583765ca620 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583764cc4a0 con 0x5583765caf40 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19979d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55837652b930 con 0x5583765d5800 2024-04-25T21:01:31.535 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc195020640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19481f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765caf40 0x5583765d52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc195020640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765caa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38248/0 (socket says 172.21.15.96:38248) 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc195020640 1 -- 172.21.15.96:0/3001076278 learned_addr learned my addr 172.21.15.96:0/3001076278 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc187fff640 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765d5800 0x5583765d7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc185ffb640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc1780a3dc0 0x7fc1780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19481f640 1 -- 172.21.15.96:0/3001076278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765d5800 msgr2=0x5583765d7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19481f640 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765d5800 0x5583765d7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19481f640 1 -- 172.21.15.96:0/3001076278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 msgr2=0x5583765caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19481f640 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc19481f640 1 -- 172.21.15.96:0/3001076278 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558376242210 con 0x5583765caf40 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.532+0000 7fc195020640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc1780a3dc0 0x7fc1780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.536 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.533+0000 7fc185ffb640 1 -- 172.21.15.96:0/978370603 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fc18c0d0750 con 0x7fc1901021d0 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.533+0000 7fc19481f640 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765caf40 0x5583765d52c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0cdf10 tx=0x7fc18c06b110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20ccac5437597139 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.533+0000 7fc195020640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc1780a3dc0 0x7fc1780a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1880105f0 tx=0x7fc188004a40 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.533+0000 7fc176ffd640 1 -- 172.21.15.96:0/3001076278 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c09c0c0 con 0x5583765caf40 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.533+0000 7fc176ffd640 1 -- 172.21.15.96:0/3001076278 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c09ce60 con 0x5583765caf40 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.533+0000 7fc176ffd640 1 -- 172.21.15.96:0/3001076278 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c09c760 con 0x5583765caf40 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 -- 172.21.15.96:0/3001076278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765caf40 msgr2=0x5583765d52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765caf40 0x5583765d52c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc18c0cdf10 tx=0x7fc18c06b110 comp rx=0 tx=0).stop 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 -- 172.21.15.96:0/3001076278 shutdown_connections 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765d5800 0x5583765d7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765caf40 0x5583765d52c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 --2- 172.21.15.96:0/3001076278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 -- 172.21.15.96:0/3001076278 >> 172.21.15.96:0/3001076278 conn(0x5583765c6b30 msgr2=0x5583765c6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 -- 172.21.15.96:0/3001076278 shutdown_connections 2024-04-25T21:01:31.537 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 -- 172.21.15.96:0/3001076278 wait complete. 2024-04-25T21:01:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 -- start start 2024-04-25T21:01:31.538 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.534+0000 7fc19979d740 1 Processor -- start 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765caf40 0x5583765d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765d28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583765d8370 con 0x5583765ca620 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5583764ca0b0 con 0x5583765d5800 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5583765d84f0 con 0x5583765caf40 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 10 client.0 _ll_register_callbacks cb 0x5583764b7d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:31.541 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765d50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38250/0 (socket says 172.21.15.96:38250) 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 -- 172.21.15.96:0/3561392392 learned_addr learned my addr 172.21.15.96:0/3561392392 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc195020640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765d28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc187fff640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765caf40 0x5583765d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 -- 172.21.15.96:0/3561392392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765caf40 msgr2=0x5583765d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765caf40 0x5583765d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 -- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 msgr2=0x5583765d28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765d28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 -- 172.21.15.96:0/3561392392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837630e620 con 0x5583765ca620 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19481f640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765d50a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc18c09d010 tx=0x7fc18c0d0a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47d38bc003aaf5c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc16ffff640 1 -- 172.21.15.96:0/3561392392 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18c01c070 con 0x5583765ca620 2024-04-25T21:01:31.542 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc16ffff640 1 -- 172.21.15.96:0/3561392392 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fc18c0b0ec0 con 0x5583765ca620 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc18c09ce60 con 0x5583765ca620 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc16ffff640 1 -- 172.21.15.96:0/3561392392 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc18c09c0c0 con 0x5583765ca620 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 10 client.15735 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.535+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5583765e41f0 con 0x5583765ca620 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc16ffff640 1 -- 172.21.15.96:0/3561392392 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc18c09ce60 con 0x5583765ca620 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc16ffff640 1 -- 172.21.15.96:0/3561392392 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc18c0ce190 con 0x5583765ca620 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc19979d740 10 client.15735 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc19979d740 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc19979d740 20 client.15735 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc19979d740 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5583765e5540 0x5583765e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc19979d740 4 client.15735 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837653d390 con 0x5583765e5540 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.536+0000 7fc187fff640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5583765e5540 0x5583765e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.537+0000 7fc16ffff640 10 client.15735 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.537+0000 7fc187fff640 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5583765e5540 0x5583765e7930 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e93fa9c200491978 server_cookie=dfca91f559424782 in_seq=0 out_seq=0 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc16ffff640 1 -- 172.21.15.96:0/3561392392 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55837653d390 con 0x5583765e5540 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc16ffff640 10 client.15735 handle_command_reply: tid=0 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 client.15735 shutdown 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 msgr2=0x5583765d50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765d50a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc18c09d010 tx=0x7fc18c0d0a20 comp rx=0 tx=0).stop 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 shutdown_connections 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5583765e5540 0x5583765e7930 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765d28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc16ffff640 0 client.15735 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765caf40 0x5583765d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 --2- 172.21.15.96:0/3561392392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765ca620 0x5583765d50a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 >> 172.21.15.96:0/3561392392 conn(0x5583765c6b30 msgr2=0x5583765cc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 shutdown_connections 2024-04-25T21:01:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 1 -- 172.21.15.96:0/3561392392 wait complete. 2024-04-25T21:01:31.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.538+0000 7fc19979d740 20 client.15735 trim_cache size 0 max 16384 2024-04-25T21:01:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.556+0000 7fc19979d740 1 Processor -- start 2024-04-25T21:01:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.556+0000 7fc19979d740 1 -- start start 2024-04-25T21:01:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19979d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765d4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19979d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765ecfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19979d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765ed500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19979d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901b0300 con 0x5583765caf40 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19979d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901adb70 con 0x5583765d5800 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19979d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ab9e0 con 0x5583765ca620 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765ecfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765ecfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38256/0 (socket says 172.21.15.96:38256) 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 -- 172.21.15.96:0/1140019234 learned_addr learned my addr 172.21.15.96:0/1140019234 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19481f640 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765d4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc195020640 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765ed500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 -- 172.21.15.96:0/1140019234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 msgr2=0x5583765d4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765d4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 -- 172.21.15.96:0/1140019234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 msgr2=0x5583765ed500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765ed500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 -- 172.21.15.96:0/1140019234 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583765d2c30 con 0x5583765caf40 2024-04-25T21:01:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc19481f640 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc187fff640 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765ecfc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc180000c00 tx=0x7fc180005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00ff1a18d454590 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.557+0000 7fc1767fc640 1 -- 172.21.15.96:0/1140019234 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc18004a030 con 0x5583765caf40 2024-04-25T21:01:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc1767fc640 1 -- 172.21.15.96:0/1140019234 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc180031e50 con 0x5583765caf40 2024-04-25T21:01:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc1767fc640 1 -- 172.21.15.96:0/1140019234 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180035040 con 0x5583765caf40 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 -- 172.21.15.96:0/1140019234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 msgr2=0x5583765ecfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765ecfc0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc180000c00 tx=0x7fc180005db0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 -- 172.21.15.96:0/1140019234 shutdown_connections 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765ed500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765ecfc0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 --2- 172.21.15.96:0/1140019234 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765d4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 -- 172.21.15.96:0/1140019234 >> 172.21.15.96:0/1140019234 conn(0x5583765c6b30 msgr2=0x5583765c5cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 -- 172.21.15.96:0/1140019234 shutdown_connections 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.558+0000 7fc19979d740 1 -- 172.21.15.96:0/1140019234 wait complete. 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 -- start start 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 Processor -- start 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765dd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765dd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ac030 con 0x5583765caf40 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ad4a0 con 0x5583765d5800 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc1901ac700 con 0x5583765ca620 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19979d740 10 client.0 _ll_register_callbacks cb 0x7fc18c0988c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765dd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc187fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765dd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:38262/0 (socket says 172.21.15.96:38262) 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc187fff640 1 -- 172.21.15.96:0/214774485 learned_addr learned my addr 172.21.15.96:0/214774485 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc19481f640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765dd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.559+0000 7fc195020640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc187fff640 1 -- 172.21.15.96:0/214774485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 msgr2=0x5583765dd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc187fff640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765dd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc187fff640 1 -- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 msgr2=0x5583765da8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc187fff640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765da8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc187fff640 1 -- 172.21.15.96:0/214774485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583765db0d0 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc19481f640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc195020640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc187fff640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765dd9a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc180009230 tx=0x7fc180009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b758601b3c03f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc16ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc16ffff640 1 -- 172.21.15.96:0/214774485 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc180035050 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc16ffff640 1 -- 172.21.15.96:0/214774485 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fc1800318b0 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc180031e50 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc16ffff640 1 -- 172.21.15.96:0/214774485 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc18005f050 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc19979d740 10 client.15741 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.560+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55837664b2d0 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc16ffff640 1 -- 172.21.15.96:0/214774485 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc180031e50 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc16ffff640 1 -- 172.21.15.96:0/214774485 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7fc180046840 con 0x5583765caf40 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19979d740 10 client.15741 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19979d740 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19979d740 20 client.15741 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19979d740 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55837664b9d0 0x55837664ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19979d740 4 client.15741 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558376537200 con 0x55837664b9d0 2024-04-25T21:01:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19481f640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55837664b9d0 0x55837664ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc16ffff640 10 client.15741 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.561+0000 7fc19481f640 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55837664b9d0 0x55837664ddf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f6c9ca209a664d server_cookie=bff49a9e27d69f68 in_seq=0 out_seq=0 2024-04-25T21:01:31.585 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.584+0000 7fc16ffff640 1 -- 172.21.15.96:0/214774485 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558376537200 con 0x55837664b9d0 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.584+0000 7fc16ffff640 10 client.15741 handle_command_reply: tid=0 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.584+0000 7fc19979d740 1 client.15741 shutdown 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.584+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 msgr2=0x5583765dd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.584+0000 7fc19979d740 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765dd9a0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fc180009230 tx=0x7fc180009520 comp rx=0 tx=0).stop 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 shutdown_connections 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55837664b9d0 0x55837664ddf0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5583765d5800 0x5583765da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583765caf40 0x5583765dd9a0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 --2- 172.21.15.96:0/214774485 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5583765ca620 0x5583765dd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 >> 172.21.15.96:0/214774485 conn(0x5583765c6b30 msgr2=0x5583765d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc16ffff640 0 client.15741 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 shutdown_connections 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 1 -- 172.21.15.96:0/214774485 wait complete. 2024-04-25T21:01:31.586 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.585+0000 7fc19979d740 20 client.15741 trim_cache size 0 max 16384 2024-04-25T21:01:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:01:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:01:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:01:31.587 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc1780a3dc0 msgr2=0x7fc1780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc1780a3dc0 0x7fc1780a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc1880105f0 tx=0x7fc188004a40 comp rx=0 tx=0).stop 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901021d0 msgr2=0x7fc1901986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901021d0 0x7fc1901986e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00e9f0 tx=0x7fc18c00eeb0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 shutdown_connections 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc1780a3dc0 0x7fc1780a6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc19019cfa0 0x7fc19019d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1901072a0 0x7fc190198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 --2- 172.21.15.96:0/978370603 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc1901021d0 0x7fc1901986e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 >> 172.21.15.96:0/978370603 conn(0x7fc1900793e0 msgr2=0x7fc1900ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 shutdown_connections 2024-04-25T21:01:31.588 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:31.586+0000 7fc196a83640 1 -- 172.21.15.96:0/978370603 wait complete. 2024-04-25T21:01:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:32 smithi096 ceph-mon[29288]: pgmap v120: 97 pgs: 97 active+clean; 55 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.6 MiB/s wr, 712 op/s 2024-04-25T21:01:32.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:32 smithi097 ceph-mon[32667]: pgmap v120: 97 pgs: 97 active+clean; 55 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.6 MiB/s wr, 712 op/s 2024-04-25T21:01:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:32 smithi139 ceph-mon[33027]: pgmap v120: 97 pgs: 97 active+clean; 55 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.6 MiB/s wr, 712 op/s 2024-04-25T21:01:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:34 smithi096 ceph-mon[29288]: pgmap v121: 97 pgs: 97 active+clean; 55 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.2 MiB/s wr, 505 op/s 2024-04-25T21:01:34.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:34 smithi097 ceph-mon[32667]: pgmap v121: 97 pgs: 97 active+clean; 55 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.2 MiB/s wr, 505 op/s 2024-04-25T21:01:34.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:34 smithi139 ceph-mon[33027]: pgmap v121: 97 pgs: 97 active+clean; 55 MiB data, 867 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 3.2 MiB/s wr, 505 op/s 2024-04-25T21:01:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:36 smithi096 ceph-mon[29288]: pgmap v122: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 884 op/s 2024-04-25T21:01:36.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:36 smithi097 ceph-mon[32667]: pgmap v122: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 884 op/s 2024-04-25T21:01:36.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:36 smithi139 ceph-mon[33027]: pgmap v122: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 5.4 MiB/s wr, 884 op/s 2024-04-25T21:01:37.848 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x9ff773 2024-04-25T21:01:38.280 INFO:tasks.workunit.client.0.smithi096.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-04-25T21:01:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:38 smithi096 ceph-mon[29288]: pgmap v123: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2024-04-25T21:01:38.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:38 smithi097 ceph-mon[32667]: pgmap v123: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2024-04-25T21:01:38.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:38 smithi139 ceph-mon[33027]: pgmap v123: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2024-04-25T21:01:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:40 smithi096 ceph-mon[29288]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2024-04-25T21:01:40.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:40 smithi097 ceph-mon[32667]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2024-04-25T21:01:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:40 smithi139 ceph-mon[33027]: pgmap v124: 97 pgs: 97 active+clean; 48 MiB data, 858 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 MiB/s rd, 4.0 MiB/s wr, 697 op/s 2024-04-25T21:01:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:42 smithi096 ceph-mon[29288]: pgmap v125: 97 pgs: 97 active+clean; 55 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.0 MiB/s wr, 1.04k op/s 2024-04-25T21:01:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:42 smithi097 ceph-mon[32667]: pgmap v125: 97 pgs: 97 active+clean; 55 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.0 MiB/s wr, 1.04k op/s 2024-04-25T21:01:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:42 smithi139 ceph-mon[33027]: pgmap v125: 97 pgs: 97 active+clean; 55 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.0 MiB/s wr, 1.04k op/s 2024-04-25T21:01:44.683 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 -- 172.21.15.96:0/216239738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06b080 msgr2=0x7f186c10e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 --2- 172.21.15.96:0/216239738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06b080 0x7f186c10e710 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1860009900 tx=0x7f186002a480 comp rx=0 tx=0).stop 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 -- 172.21.15.96:0/216239738 shutdown_connections 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 --2- 172.21.15.96:0/216239738 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c10ec50 0x7f186c111040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 --2- 172.21.15.96:0/216239738 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06b080 0x7f186c10e710 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 --2- 172.21.15.96:0/216239738 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06a760 0x7f186c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 -- 172.21.15.96:0/216239738 >> 172.21.15.96:0/216239738 conn(0x7f186c077cc0 msgr2=0x7f186c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 -- 172.21.15.96:0/216239738 shutdown_connections 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.683+0000 7f1871f44640 1 -- 172.21.15.96:0/216239738 wait complete. 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 Processor -- start 2024-04-25T21:01:44.684 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 -- start start 2024-04-25T21:01:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 0x7f186c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06b080 0x7f186c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c10ec50 0x7f186c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f186c114470 con 0x7f186c06a760 2024-04-25T21:01:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1142f0 con 0x7f186c10ec50 2024-04-25T21:01:44.686 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f1871f44640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1145f0 con 0x7f186c06b080 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.684+0000 7f186b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 0x7f186c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 0x7f186c1a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52810/0 (socket says 172.21.15.96:52810) 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 -- 172.21.15.96:0/3417078193 learned_addr learned my addr 172.21.15.96:0/3417078193 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186affd640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06b080 0x7f186c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186bfff640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c10ec50 0x7f186c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 -- 172.21.15.96:0/3417078193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06b080 msgr2=0x7f186c1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06b080 0x7f186c1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 -- 172.21.15.96:0/3417078193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c10ec50 msgr2=0x7f186c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c10ec50 0x7f186c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 -- 172.21.15.96:0/3417078193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f186c19af70 con 0x7f186c06a760 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186affd640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06b080 0x7f186c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f186b7fe640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 0x7f186c1a05a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e9f0 tx=0x7f185c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.687 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f1868ff9640 1 -- 172.21.15.96:0/3417078193 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c00cd80 con 0x7f186c06a760 2024-04-25T21:01:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f1868ff9640 1 -- 172.21.15.96:0/3417078193 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c0054d0 con 0x7f186c06a760 2024-04-25T21:01:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f186c19b260 con 0x7f186c06a760 2024-04-25T21:01:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f1868ff9640 1 -- 172.21.15.96:0/3417078193 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c010690 con 0x7f186c06a760 2024-04-25T21:01:44.688 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.685+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f186c1a7520 con 0x7f186c06a760 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.688+0000 7f1868ff9640 1 -- 172.21.15.96:0/3417078193 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f185c004e90 con 0x7f186c06a760 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 Processor -- start 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 -- start start 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1868ff9640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f18540a3e40 0x7f18540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2d0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 0x56371b2db2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b2ddbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56371b235970 con 0x56371b2db800 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56371b1d24a0 con 0x56371b2d0620 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1874c63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56371b231930 con 0x56371b2d0f40 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f1868ff9640 1 -- 172.21.15.96:0/3417078193 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f185c0d0b00 con 0x7f186c06a760 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 0x56371b2db2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.690 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2d0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b2ddbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186b7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2d0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:39514/0 (socket says 172.21.15.96:39514) 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186affd640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 0x56371b2db2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:45316/0 (socket says 172.21.15.96:45316) 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186b7fe640 1 -- 172.21.15.96:0/4028222909 learned_addr learned my addr 172.21.15.96:0/4028222909 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.689+0000 7f186bfff640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f18540a3e40 0x7f18540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186b7fe640 1 -- 172.21.15.96:0/4028222909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 msgr2=0x56371b2db2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186b7fe640 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 0x56371b2db2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186b7fe640 1 -- 172.21.15.96:0/4028222909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 msgr2=0x56371b2ddbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186b7fe640 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b2ddbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186b7fe640 1 -- 172.21.15.96:0/4028222909 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371af48210 con 0x56371b2d0620 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186affd640 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 0x56371b2db2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186bfff640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f18540a3e40 0x7f18540a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1864005c40 tx=0x7f1864009b80 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f186b7fe640 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2d0a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f185c098f70 tx=0x7f185c098ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b490464085aa168c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.691 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f1851ffb640 1 -- 172.21.15.96:0/4028222909 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c070a80 con 0x56371b2d0620 2024-04-25T21:01:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f1851ffb640 1 -- 172.21.15.96:0/4028222909 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c067970 con 0x56371b2d0620 2024-04-25T21:01:44.692 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.690+0000 7f1851ffb640 1 -- 172.21.15.96:0/4028222909 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f185c0a5780 con 0x56371b2d0620 2024-04-25T21:01:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 -- 172.21.15.96:0/4028222909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 msgr2=0x56371b2d0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.696 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2d0a00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f185c098f70 tx=0x7f185c098ee0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 -- 172.21.15.96:0/4028222909 shutdown_connections 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b2ddbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0f40 0x56371b2db2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 --2- 172.21.15.96:0/4028222909 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2d0a00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 -- 172.21.15.96:0/4028222909 >> 172.21.15.96:0/4028222909 conn(0x56371b2ccb30 msgr2=0x56371b2ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 -- 172.21.15.96:0/4028222909 shutdown_connections 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 -- 172.21.15.96:0/4028222909 wait complete. 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.691+0000 7f1874c63740 1 -- start start 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 Processor -- start 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2db1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2d84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.697 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2db800 0x56371b2d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56371b2de3d0 con 0x56371b2d0f40 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56371b1d00b0 con 0x56371b2db800 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56371b2de550 con 0x56371b2d0620 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f1874c63740 10 client.0 _ll_register_callbacks cb 0x56371b1bdd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2d84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186bfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2db800 0x56371b2d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2d84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52832/0 (socket says 172.21.15.96:52832) 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186affd640 1 -- 172.21.15.96:0/12751176 learned_addr learned my addr 172.21.15.96:0/12751176 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186b7fe640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2db1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186bfff640 1 -- 172.21.15.96:0/12751176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 msgr2=0x56371b2db1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186bfff640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2db1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186bfff640 1 -- 172.21.15.96:0/12751176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 msgr2=0x56371b2d84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186bfff640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2d84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.698 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186bfff640 1 -- 172.21.15.96:0/12751176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371b014620 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186b7fe640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2db1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.692+0000 7f186affd640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2d84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f186bfff640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2db800 0x56371b2d8ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x56371b2cdcc0 tx=0x7f1864005bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8925c8757a826ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f184affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f184affd640 1 -- 172.21.15.96:0/12751176 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186401f860 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f184affd640 1 -- 172.21.15.96:0/12751176 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f186401fa00 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f185c067970 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f184affd640 1 -- 172.21.15.96:0/12751176 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f186401fc00 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f1874c63740 10 client.25226 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56371b2ea360 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f184affd640 1 -- 172.21.15.96:0/12751176 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f185c067970 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.693+0000 7f184affd640 1 -- 172.21.15.96:0/12751176 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1864030030 con 0x56371b2db800 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f1874c63740 10 client.25226 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:44.699 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f1874c63740 10 client.25226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f1874c63740 20 client.25226 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f1874c63740 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b2ebd10 0x56371b2ee150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f1874c63740 4 client.25226 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56371b243390 con 0x56371b2ebd10 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f186b7fe640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b2ebd10 0x56371b2ee150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f184affd640 10 client.25226 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.694+0000 7f186b7fe640 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b2ebd10 0x56371b2ee150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30e864dc0eca710f server_cookie=9970a8a000dd072f in_seq=0 out_seq=0 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f184affd640 1 -- 172.21.15.96:0/12751176 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56371b243390 con 0x56371b2ebd10 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f184affd640 10 client.25226 handle_command_reply: tid=0 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 client.25226 shutdown 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2db800 msgr2=0x56371b2d8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2db800 0x56371b2d8ac0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x56371b2cdcc0 tx=0x7f1864005bd0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 shutdown_connections 2024-04-25T21:01:44.700 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b2ebd10 0x56371b2ee150 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2db800 0x56371b2d8ac0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f184affd640 0 client.25226 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2d84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 --2- 172.21.15.96:0/12751176 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2db1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 >> 172.21.15.96:0/12751176 conn(0x56371b2ccb30 msgr2=0x56371b2d28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 shutdown_connections 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 1 -- 172.21.15.96:0/12751176 wait complete. 2024-04-25T21:01:44.701 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.695+0000 7f1874c63740 20 client.25226 trim_cache size 0 max 16384 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 Processor -- start 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 -- start start 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2dacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2f3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2db800 0x56371b2f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1b23d0 con 0x56371b2d0f40 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1afc40 con 0x56371b2d0620 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f1874c63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ada90 con 0x56371b2db800 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f186affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f186affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52844/0 (socket says 172.21.15.96:52844) 2024-04-25T21:01:44.713 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f186affd640 1 -- 172.21.15.96:0/3493905191 learned_addr learned my addr 172.21.15.96:0/3493905191 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f186bfff640 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2db800 0x56371b2f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.712+0000 7f186b7fe640 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2dacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f186affd640 1 -- 172.21.15.96:0/3493905191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2db800 msgr2=0x56371b2f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f186affd640 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2db800 0x56371b2f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f186affd640 1 -- 172.21.15.96:0/3493905191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 msgr2=0x56371b2dacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f186affd640 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2dacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f186affd640 1 -- 172.21.15.96:0/3493905191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56371b2d9420 con 0x56371b2d0f40 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f186affd640 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2f3800 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1860009230 tx=0x7f1860004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8250b28376aef414 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f18517fa640 1 -- 172.21.15.96:0/3493905191 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860035040 con 0x56371b2d0f40 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f18517fa640 1 -- 172.21.15.96:0/3493905191 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1860004770 con 0x56371b2d0f40 2024-04-25T21:01:44.714 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.713+0000 7f18517fa640 1 -- 172.21.15.96:0/3493905191 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18600038e0 con 0x56371b2d0f40 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 -- 172.21.15.96:0/3493905191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 msgr2=0x56371b2f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2f3800 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1860009230 tx=0x7f1860004610 comp rx=0 tx=0).stop 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 -- 172.21.15.96:0/3493905191 shutdown_connections 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2db800 0x56371b2f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2d0f40 0x56371b2f3800 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 --2- 172.21.15.96:0/3493905191 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0620 0x56371b2dacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 -- 172.21.15.96:0/3493905191 >> 172.21.15.96:0/3493905191 conn(0x56371b2ccb30 msgr2=0x56371b2d3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 -- 172.21.15.96:0/3493905191 shutdown_connections 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 -- 172.21.15.96:0/3493905191 wait complete. 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 -- start start 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.714+0000 7f1874c63740 1 Processor -- start 2024-04-25T21:01:44.718 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2f0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 0x56371b2f0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b3501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ae100 con 0x56371b2db800 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1af570 con 0x56371b2d0f40 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f186c1ae7d0 con 0x56371b2d0620 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f1874c63740 10 client.0 _ll_register_callbacks cb 0x7f1864003c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b3501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 0x56371b2f0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 0x56371b2f0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:39544/0 (socket says 172.21.15.96:39544) 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b3501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:52854/0 (socket says 172.21.15.96:52854) 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 -- 172.21.15.96:0/2738507350 learned_addr learned my addr 172.21.15.96:0/2738507350 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186b7fe640 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2f0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 -- 172.21.15.96:0/2738507350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 msgr2=0x56371b2f0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2f0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.719 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 -- 172.21.15.96:0/2738507350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 msgr2=0x56371b3501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b3501b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 -- 172.21.15.96:0/2738507350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56371b3506f0 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f186affd640 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 0x56371b2f0f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1860006010 tx=0x7f1860004b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1a3a69eb7cc43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f184affd640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.715+0000 7f184affd640 1 -- 172.21.15.96:0/2738507350 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1860035070 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1860004770 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f184affd640 1 -- 172.21.15.96:0/2738507350 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f18600035f0 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f184affd640 1 -- 172.21.15.96:0/2738507350 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18600055a0 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 10 client.25232 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56371b351970 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f184affd640 1 -- 172.21.15.96:0/2738507350 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1860004770 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f184affd640 1 -- 172.21.15.96:0/2738507350 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f1860057050 con 0x56371b2d0f40 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 10 client.25232 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:44.720 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 10 client.25232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 20 client.25232 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b352050 0x56371b354470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 4 client.25232 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56371b23d200 con 0x56371b352050 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.716+0000 7f186b7fe640 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b352050 0x56371b354470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.717+0000 7f184affd640 10 client.25232 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:44.721 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.717+0000 7f186b7fe640 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b352050 0x56371b354470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f567331ebf19ce7 server_cookie=3acd763822e3d573 in_seq=0 out_seq=0 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f184affd640 1 -- 172.21.15.96:0/2738507350 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56371b23d200 con 0x56371b352050 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f184affd640 10 client.25232 handle_command_reply: tid=0 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 client.25232 shutdown 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 msgr2=0x56371b2f0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 0x56371b2f0f90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1860006010 tx=0x7f1860004b50 comp rx=0 tx=0).stop 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 shutdown_connections 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56371b352050 0x56371b354470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56371b2db800 0x56371b3501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56371b2d0f40 0x56371b2f0f90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 --2- 172.21.15.96:0/2738507350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56371b2d0620 0x56371b2f0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 >> 172.21.15.96:0/2738507350 conn(0x56371b2ccb30 msgr2=0x56371b2cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f184affd640 0 client.25232 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 shutdown_connections 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 1 -- 172.21.15.96:0/2738507350 wait complete. 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.737+0000 7f1874c63740 20 client.25232 trim_cache size 0 max 16384 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:01:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:01:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f18540a3e40 msgr2=0x7f18540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f18540a3e40 0x7f18540a6300 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1864005c40 tx=0x7f1864009b80 comp rx=0 tx=0).stop 2024-04-25T21:01:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 msgr2=0x7f186c1a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:44.739 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 0x7f186c1a05a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f185c00e9f0 tx=0x7f185c00edf0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 shutdown_connections 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f18540a3e40 0x7f18540a6300 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f186c10ec50 0x7f186c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f186c06b080 0x7f186c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 --2- 172.21.15.96:0/3417078193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f186c06a760 0x7f186c1a05a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 >> 172.21.15.96:0/3417078193 conn(0x7f186c077cc0 msgr2=0x7f186c110140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 shutdown_connections 2024-04-25T21:01:44.740 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:44.738+0000 7f1871f44640 1 -- 172.21.15.96:0/3417078193 wait complete. 2024-04-25T21:01:44.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:44 smithi096 ceph-mon[29288]: pgmap v126: 97 pgs: 97 active+clean; 55 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 721 op/s 2024-04-25T21:01:44.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:44 smithi097 ceph-mon[32667]: pgmap v126: 97 pgs: 97 active+clean; 55 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 721 op/s 2024-04-25T21:01:44.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:44 smithi139 ceph-mon[33027]: pgmap v126: 97 pgs: 97 active+clean; 55 MiB data, 911 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.2 MiB/s wr, 721 op/s 2024-04-25T21:01:45.364 INFO:tasks.workunit.client.0.smithi096.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-04-25T21:01:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:46 smithi096 ceph-mon[29288]: pgmap v127: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.0 MiB/s wr, 1.04k op/s 2024-04-25T21:01:46.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:46 smithi097 ceph-mon[32667]: pgmap v127: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.0 MiB/s wr, 1.04k op/s 2024-04-25T21:01:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:46 smithi139 ceph-mon[33027]: pgmap v127: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 6.0 MiB/s wr, 1.04k op/s 2024-04-25T21:01:47.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:47 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:47 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:47.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:47 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:01:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:48 smithi096 ceph-mon[29288]: pgmap v128: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T21:01:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:48 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:48 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:48 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:01:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:48 smithi097 ceph-mon[32667]: pgmap v128: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T21:01:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:48 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:48 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:48 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:01:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:48 smithi139 ceph-mon[33027]: pgmap v128: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T21:01:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:48 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:01:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:48 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:01:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:48 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:01:49.280 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x9ffb01 2024-04-25T21:01:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:50 smithi096 ceph-mon[29288]: pgmap v129: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T21:01:50.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:50 smithi097 ceph-mon[32667]: pgmap v129: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T21:01:50.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:50 smithi139 ceph-mon[33027]: pgmap v129: 97 pgs: 97 active+clean; 56 MiB data, 978 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.8 MiB/s wr, 659 op/s 2024-04-25T21:01:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:52 smithi096 ceph-mon[29288]: pgmap v130: 97 pgs: 97 active+clean; 49 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 955 op/s 2024-04-25T21:01:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:52 smithi097 ceph-mon[32667]: pgmap v130: 97 pgs: 97 active+clean; 49 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 955 op/s 2024-04-25T21:01:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:52 smithi139 ceph-mon[33027]: pgmap v130: 97 pgs: 97 active+clean; 49 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 955 op/s 2024-04-25T21:01:52.913 INFO:tasks.workunit.client.0.smithi096.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-04-25T21:01:54.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:54 smithi096 ceph-mon[29288]: pgmap v131: 97 pgs: 97 active+clean; 49 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 613 op/s 2024-04-25T21:01:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:54 smithi097 ceph-mon[32667]: pgmap v131: 97 pgs: 97 active+clean; 49 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 613 op/s 2024-04-25T21:01:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:54 smithi139 ceph-mon[33027]: pgmap v131: 97 pgs: 97 active+clean; 49 MiB data, 987 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 3.6 MiB/s wr, 613 op/s 2024-04-25T21:01:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:56 smithi096 ceph-mon[29288]: pgmap v132: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 912 op/s 2024-04-25T21:01:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:56 smithi097 ceph-mon[32667]: pgmap v132: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 912 op/s 2024-04-25T21:01:56.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:56 smithi139 ceph-mon[33027]: pgmap v132: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 912 op/s 2024-04-25T21:01:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3168907740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101aa0 msgr2=0x7f9e98105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.847 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3168907740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101aa0 0x7f9e98105a90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009900 tx=0x7f9e8802a410 comp rx=0 tx=0).stop 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3168907740 shutdown_connections 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3168907740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98106170 0x7f9e9810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3168907740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101aa0 0x7f9e98105a90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3168907740 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e981010f0 0x7f9e981014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3168907740 >> 172.21.15.96:0/3168907740 conn(0x7f9e98077fd0 msgr2=0x7f9e980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3168907740 shutdown_connections 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.832+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3168907740 wait complete. 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 Processor -- start 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 -- start start 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 0x7f9e981994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98106170 0x7f9e98199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e98199fc0 0x7f9e981a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981110b0 con 0x7f9e981010f0 2024-04-25T21:01:57.848 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98110f30 con 0x7f9e98106170 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9f42a640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98111230 con 0x7f9e98199fc0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 0x7f9e981994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 0x7f9e981994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57214/0 (socket says 172.21.15.96:57214) 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3339239900 learned_addr learned my addr 172.21.15.96:0/3339239900 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98106170 0x7f9e98199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e98199fc0 0x7f9e981a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3339239900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e98199fc0 msgr2=0x7f9e981a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e98199fc0 0x7f9e981a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3339239900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98106170 msgr2=0x7f9e98199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98106170 0x7f9e98199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.833+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3339239900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e981a4d40 con 0x7f9e981010f0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.834+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 0x7f9e981994f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400e9f0 tx=0x7f9e9400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.834+0000 7f9e8e7fc640 1 -- 172.21.15.96:0/3339239900 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9400cd80 con 0x7f9e981010f0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.834+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981a4fd0 con 0x7f9e981010f0 2024-04-25T21:01:57.849 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.834+0000 7f9e8e7fc640 1 -- 172.21.15.96:0/3339239900 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e940054d0 con 0x7f9e981010f0 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.834+0000 7f9e8e7fc640 1 -- 172.21.15.96:0/3339239900 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94010690 con 0x7f9e981010f0 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.834+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e981a54a0 con 0x7f9e981010f0 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.836+0000 7f9e8e7fc640 1 -- 172.21.15.96:0/3339239900 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e94004e90 con 0x7f9e981010f0 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 Processor -- start 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 -- start start 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fd9f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda06000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5635fd959970 con 0x5635fd9f4e40 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635fd8f64a0 con 0x5635fda03c10 2024-04-25T21:01:57.850 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9ea221a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5635fd955930 con 0x5635fd9f4520 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9e8e7fc640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9e840a3e40 0x7f9e840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.837+0000 7f9e9d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9c9c5640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda036d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57216/0 (socket says 172.21.15.96:57216) 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d9c7640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fd9f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3610859620 learned_addr learned my addr 172.21.15.96:0/3610859620 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9e840a3e40 0x7f9e840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e8e7fc640 1 -- 172.21.15.96:0/3339239900 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f9e940d0be0 con 0x7f9e981010f0 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3610859620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 msgr2=0x5635fd9f4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fd9f4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3610859620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 msgr2=0x5635fda06000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.851 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda06000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 -- 172.21.15.96:0/3610859620 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fd66c210 con 0x5635fd9f4e40 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda036d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9e94098fa0 tx=0x7f9e9409c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83330a190aefd3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9e840a3e40 0x7f9e840a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005fd0 tx=0x7f9e90009500 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e837fe640 1 -- 172.21.15.96:0/3610859620 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9409cb90 con 0x5635fd9f4e40 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e837fe640 1 -- 172.21.15.96:0/3610859620 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e940a0f40 con 0x5635fd9f4e40 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.838+0000 7f9e837fe640 1 -- 172.21.15.96:0/3610859620 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94099200 con 0x5635fd9f4e40 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 -- 172.21.15.96:0/3610859620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 msgr2=0x5635fda036d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda036d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9e94098fa0 tx=0x7f9e9409c620 comp rx=0 tx=0).stop 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 -- 172.21.15.96:0/3610859620 shutdown_connections 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda06000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda036d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3610859620 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fd9f4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 -- 172.21.15.96:0/3610859620 >> 172.21.15.96:0/3610859620 conn(0x5635fd9f0b30 msgr2=0x5635fd9f0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 -- 172.21.15.96:0/3610859620 shutdown_connections 2024-04-25T21:01:57.852 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.839+0000 7f9ea221a740 1 -- 172.21.15.96:0/3610859620 wait complete. 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 -- start start 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 Processor -- start 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda00d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5635fda067e0 con 0x5635fd9f4e40 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5635fd8f40b0 con 0x5635fda03c10 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5635fda06960 con 0x5635fd9f4520 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9ea221a740 10 client.0 _ll_register_callbacks cb 0x5635fd8e1d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9e9c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9e9c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda007e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57220/0 (socket says 172.21.15.96:57220) 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9e9d9c7640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda00d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.853 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3229539207 learned_addr learned my addr 172.21.15.96:0/3229539207 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.840+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3229539207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 msgr2=0x5635fda034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 msgr2=0x5635fda00d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda00d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3229539207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fd738620 con 0x5635fd9f4e40 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda00d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda007e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009900 tx=0x7f9e88032de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d890a3a39d8f6664 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e80ff9640 1 -- 172.21.15.96:0/3229539207 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88035040 con 0x5635fd9f4e40 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e80ff9640 1 -- 172.21.15.96:0/3229539207 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f9e88031760 con 0x5635fd9f4e40 2024-04-25T21:01:57.854 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e940a0f40 con 0x5635fd9f4e40 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e80ff9640 1 -- 172.21.15.96:0/3229539207 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e88004040 con 0x5635fd9f4e40 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9ea221a740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635fda12700 con 0x5635fd9f4e40 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.841+0000 7f9e80ff9640 1 -- 172.21.15.96:0/3229539207 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e940a0f40 con 0x5635fd9f4e40 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9e80ff9640 1 -- 172.21.15.96:0/3229539207 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f9e8804e020 con 0x5635fd9f4e40 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9ea221a740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9ea221a740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9ea221a740 20 client.15771 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda13c30 0x5635fda16020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9ea221a740 4 client.15771 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:01:57.855 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635fd967390 con 0x5635fda13c30 2024-04-25T21:01:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda13c30 0x5635fda16020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9e80ff9640 10 client.15771 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.842+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda13c30 0x5635fda16020 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cff9598ab70f5d0 server_cookie=a320d28af33c700d in_seq=0 out_seq=0 2024-04-25T21:01:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9e80ff9640 1 -- 172.21.15.96:0/3229539207 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635fd967390 con 0x5635fda13c30 2024-04-25T21:01:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9e80ff9640 10 client.15771 handle_command_reply: tid=0 2024-04-25T21:01:57.856 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 client.15771 shutdown 2024-04-25T21:01:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 msgr2=0x5635fda007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda007e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009900 tx=0x7f9e88032de0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 shutdown_connections 2024-04-25T21:01:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda13c30 0x5635fda16020 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda00d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.857 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda007e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3229539207 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda034f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 >> 172.21.15.96:0/3229539207 conn(0x5635fd9f0b30 msgr2=0x5635fd9f69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9e80ff9640 0 client.15771 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 shutdown_connections 2024-04-25T21:01:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 1 -- 172.21.15.96:0/3229539207 wait complete. 2024-04-25T21:01:57.858 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.843+0000 7f9ea221a740 20 client.15771 trim_cache size 0 max 16384 2024-04-25T21:01:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 Processor -- start 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 -- start start 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda03020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda1b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981ae0f0 con 0x5635fd9f4e40 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981ab960 con 0x5635fda03c10 2024-04-25T21:01:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9ea221a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981a97d0 con 0x5635fd9f4520 2024-04-25T21:01:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9e9c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda03020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9e9c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda03020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57236/0 (socket says 172.21.15.96:57236) 2024-04-25T21:01:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.860+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3677882028 learned_addr learned my addr 172.21.15.96:0/3677882028 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda1b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3677882028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 msgr2=0x5635fda0f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3677882028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 msgr2=0x5635fda1b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda1b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9c9c5640 1 -- 172.21.15.96:0/3677882028 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635fd9ec2e0 con 0x5635fd9f4e40 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda03020 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009850 tx=0x7f9e8802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfbb955b09aa22e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e82ffd640 1 -- 172.21.15.96:0/3677882028 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88035070 con 0x5635fd9f4e40 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e82ffd640 1 -- 172.21.15.96:0/3677882028 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e88055070 con 0x5635fd9f4e40 2024-04-25T21:01:57.871 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.861+0000 7f9e82ffd640 1 -- 172.21.15.96:0/3677882028 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88003850 con 0x5635fd9f4e40 2024-04-25T21:01:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 -- 172.21.15.96:0/3677882028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 msgr2=0x5635fda03020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda03020 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009850 tx=0x7f9e8802a410 comp rx=0 tx=0).stop 2024-04-25T21:01:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 -- 172.21.15.96:0/3677882028 shutdown_connections 2024-04-25T21:01:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fda03c10 0x5635fda1b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fd9f4e40 0x5635fda03020 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.872 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 --2- 172.21.15.96:0/3677882028 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4520 0x5635fda0f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 -- 172.21.15.96:0/3677882028 >> 172.21.15.96:0/3677882028 conn(0x5635fd9f0b30 msgr2=0x5635fda007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 -- 172.21.15.96:0/3677882028 shutdown_connections 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 -- 172.21.15.96:0/3677882028 wait complete. 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 -- start start 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.862+0000 7f9ea221a740 1 Processor -- start 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fd9f4520 0x5635fda0bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.873 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4e40 0x5635fda0c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 0x5635fd9fca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981a9e20 con 0x5635fda03c10 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981ab290 con 0x5635fd9f4520 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981aa4f0 con 0x5635fd9f4e40 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 10 client.0 _ll_register_callbacks cb 0x7f9e88009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 0x5635fd9fca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.874 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 0x5635fd9fca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57250/0 (socket says 172.21.15.96:57250) 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d1c6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fd9f4520 0x5635fda0bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 -- 172.21.15.96:0/1492497600 learned_addr learned my addr 172.21.15.96:0/1492497600 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9c9c5640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4e40 0x5635fda0c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 -- 172.21.15.96:0/1492497600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4e40 msgr2=0x5635fda0c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4e40 0x5635fda0c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 -- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fd9f4520 msgr2=0x5635fda0bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:01:57.875 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fd9f4520 0x5635fda0bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 -- 172.21.15.96:0/1492497600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635fd9fcf90 con 0x5635fda03c10 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fd9f4520 0x5635fda0bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e9d9c7640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 0x5635fd9fca50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9e9001e920 tx=0x7f9e90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4481d33a3ca116e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e80ff9640 1 -- 172.21.15.96:0/1492497600 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90004b20 con 0x5635fda03c10 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9e80ff9640 1 -- 172.21.15.96:0/1492497600 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f9e90004cc0 con 0x5635fda03c10 2024-04-25T21:01:57.876 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.863+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e88055070 con 0x5635fda03c10 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9e80ff9640 1 -- 172.21.15.96:0/1492497600 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e90039050 con 0x5635fda03c10 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635fda799f0 con 0x5635fda03c10 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9e80ff9640 1 -- 172.21.15.96:0/1492497600 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e88055070 con 0x5635fda03c10 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9e80ff9640 1 -- 172.21.15.96:0/1492497600 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f9e9001d070 con 0x5635fda03c10 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:01:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 10 client.15777 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:01:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 20 client.15777 populate_metadata read hostname 'smithi096' 2024-04-25T21:01:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda7a280 0x5635fda7c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:01:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 4 client.15777 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:01:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635fd961200 con 0x5635fda7a280 2024-04-25T21:01:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.864+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda7a280 0x5635fda7c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:01:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.865+0000 7f9e80ff9640 10 client.15777 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.865+0000 7f9e9d1c6640 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda7a280 0x5635fda7c6a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=946c17cb376e671d server_cookie=36a6bebf1fcf9ba in_seq=0 out_seq=0 2024-04-25T21:01:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9e80ff9640 1 -- 172.21.15.96:0/1492497600 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635fd961200 con 0x5635fda7a280 2024-04-25T21:01:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9e80ff9640 10 client.15777 handle_command_reply: tid=0 2024-04-25T21:01:57.895 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 client.15777 shutdown 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 msgr2=0x5635fd9fca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 0x5635fd9fca50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9e9001e920 tx=0x7f9e90004a40 comp rx=0 tx=0).stop 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 shutdown_connections 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5635fda7a280 0x5635fda7c6a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5635fda03c10 0x5635fd9fca50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9e80ff9640 0 client.15777 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:01:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5635fd9f4e40 0x5635fda0c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 --2- 172.21.15.96:0/1492497600 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635fd9f4520 0x5635fda0bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 >> 172.21.15.96:0/1492497600 conn(0x5635fd9f0b30 msgr2=0x5635fd9fb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 shutdown_connections 2024-04-25T21:01:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.893+0000 7f9ea221a740 1 -- 172.21.15.96:0/1492497600 wait complete. 2024-04-25T21:01:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9ea221a740 20 client.15777 trim_cache size 0 max 16384 2024-04-25T21:01:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:01:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:01:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:01:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:01:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9e840a3e40 msgr2=0x7f9e840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9e840a3e40 0x7f9e840a6300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9e90005fd0 tx=0x7f9e90009500 comp rx=0 tx=0).stop 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 msgr2=0x7f9e981994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 0x7f9e981994f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400e9f0 tx=0x7f9e9400edf0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 shutdown_connections 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f9e840a3e40 0x7f9e840a6300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f9e98199fc0 0x7f9e981a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.900 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e98106170 0x7f9e98199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 --2- 172.21.15.96:0/3339239900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e981010f0 0x7f9e981994f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:01:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 >> 172.21.15.96:0/3339239900 conn(0x7f9e98077fd0 msgr2=0x7f9e980ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:01:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.894+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 shutdown_connections 2024-04-25T21:01:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:01:57.895+0000 7f9e9f42a640 1 -- 172.21.15.96:0/3339239900 wait complete. 2024-04-25T21:01:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:01:58 smithi096 ceph-mon[29288]: pgmap v133: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-04-25T21:01:58.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:01:58 smithi097 ceph-mon[32667]: pgmap v133: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-04-25T21:01:58.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:01:58 smithi139 ceph-mon[33027]: pgmap v133: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-04-25T21:02:00.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:00 smithi096 ceph-mon[29288]: pgmap v134: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-04-25T21:02:00.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:00 smithi097 ceph-mon[32667]: pgmap v134: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-04-25T21:02:00.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:00 smithi139 ceph-mon[33027]: pgmap v134: 97 pgs: 97 active+clean; 50 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.5 MiB/s wr, 595 op/s 2024-04-25T21:02:01.018 INFO:tasks.workunit.client.0.smithi096.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-04-25T21:02:01.018 INFO:tasks.workunit.client.0.smithi096.stdout:All 50000 operations completed A-OK! 2024-04-25T21:02:01.019 INFO:tasks.workunit.client.0.smithi096.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-04-25T21:02:01.019 INFO:tasks.workunit.client.0.smithi096.stdout:Seed set to 1 2024-04-25T21:02:02.122 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-04-25T21:02:02.122 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-04-25T21:02:02.125 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-04-25T21:02:02.125 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-04-25T21:02:02.125 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support clone range, disabling! 2024-04-25T21:02:02.127 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support dedupe range, disabling! 2024-04-25T21:02:02.129 INFO:tasks.workunit.client.0.smithi096.stdout:skipping zero size read 2024-04-25T21:02:02.130 INFO:tasks.workunit.client.0.smithi096.stderr:main: filesystem does not support exchange range, disabling! 2024-04-25T21:02:02.141 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x1c37b90 2024-04-25T21:02:02.148 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x5bd330d 2024-04-25T21:02:02.165 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x637bf99 2024-04-25T21:02:02.333 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x63d9b9c 2024-04-25T21:02:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:02 smithi096 ceph-mon[29288]: pgmap v135: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 882 op/s 2024-04-25T21:02:02.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:02 smithi097 ceph-mon[32667]: pgmap v135: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 882 op/s 2024-04-25T21:02:02.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:02 smithi139 ceph-mon[33027]: pgmap v135: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 882 op/s 2024-04-25T21:02:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:04 smithi096 ceph-mon[29288]: pgmap v136: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 587 op/s 2024-04-25T21:02:04.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:04 smithi097 ceph-mon[32667]: pgmap v136: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 587 op/s 2024-04-25T21:02:04.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:04 smithi139 ceph-mon[33027]: pgmap v136: 97 pgs: 97 active+clean; 42 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 587 op/s 2024-04-25T21:02:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:06 smithi096 ceph-mon[29288]: pgmap v137: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 891 op/s 2024-04-25T21:02:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:06 smithi097 ceph-mon[32667]: pgmap v137: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 891 op/s 2024-04-25T21:02:06.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:06 smithi139 ceph-mon[33027]: pgmap v137: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 4.6 MiB/s wr, 891 op/s 2024-04-25T21:02:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:08 smithi096 ceph-mon[29288]: pgmap v138: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-25T21:02:08.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:08 smithi097 ceph-mon[32667]: pgmap v138: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-25T21:02:08.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:08 smithi139 ceph-mon[33027]: pgmap v138: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-25T21:02:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:10 smithi096 ceph-mon[29288]: pgmap v139: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-25T21:02:10.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:10 smithi097 ceph-mon[32667]: pgmap v139: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-25T21:02:10.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:10 smithi139 ceph-mon[33027]: pgmap v139: 97 pgs: 97 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 2.8 MiB/s wr, 591 op/s 2024-04-25T21:02:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.995+0000 7f175d0a0640 1 -- 172.21.15.96:0/3506512083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17581021a0 msgr2=0x7f1758102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:10.996 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.995+0000 7f175d0a0640 1 --2- 172.21.15.96:0/3506512083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17581021a0 0x7f1758102580 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1748009900 tx=0x7f174802a410 comp rx=0 tx=0).stop 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 -- 172.21.15.96:0/3506512083 shutdown_connections 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 --2- 172.21.15.96:0/3506512083 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1758107270 0x7f175810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 --2- 172.21.15.96:0/3506512083 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1758102b50 0x7f1758106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 --2- 172.21.15.96:0/3506512083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17581021a0 0x7f1758102580 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 -- 172.21.15.96:0/3506512083 >> 172.21.15.96:0/3506512083 conn(0x7f17580fda00 msgr2=0x7f17580ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 -- 172.21.15.96:0/3506512083 shutdown_connections 2024-04-25T21:02:10.997 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.996+0000 7f175d0a0640 1 -- 172.21.15.96:0/3506512083 wait complete. 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 Processor -- start 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 -- start start 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 0x7f1758198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1758107270 0x7f1758198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175819cfc0 0x7f175819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17581121d0 con 0x7f175819cfc0 2024-04-25T21:02:10.999 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1758112050 con 0x7f17581021a0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175d0a0640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1758112350 con 0x7f1758107270 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 0x7f1758198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175819cfc0 0x7f175819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 0x7f1758198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:58756/0 (socket says 172.21.15.96:58756) 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f1756d9d640 1 -- 172.21.15.96:0/2047753875 learned_addr learned my addr 172.21.15.96:0/2047753875 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.997+0000 7f175659c640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1758107270 0x7f1758198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175759e640 1 -- 172.21.15.96:0/2047753875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1758107270 msgr2=0x7f1758198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175759e640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1758107270 0x7f1758198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175759e640 1 -- 172.21.15.96:0/2047753875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 msgr2=0x7f1758198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175759e640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 0x7f1758198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175759e640 1 -- 172.21.15.96:0/2047753875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f175819dab0 con 0x7f175819cfc0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f1756d9d640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 0x7f1758198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175759e640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175819cfc0 0x7f175819d470 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f175000ec50 tx=0x7f175000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f173ffff640 1 -- 172.21.15.96:0/2047753875 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750003b70 con 0x7f175819cfc0 2024-04-25T21:02:11.000 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f173ffff640 1 -- 172.21.15.96:0/2047753875 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1750003d10 con 0x7f175819cfc0 2024-04-25T21:02:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f175819dda0 con 0x7f175819cfc0 2024-04-25T21:02:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f173ffff640 1 -- 172.21.15.96:0/2047753875 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1750010830 con 0x7f175819cfc0 2024-04-25T21:02:11.001 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:10.998+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17581a5540 con 0x7f175819cfc0 2024-04-25T21:02:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.000+0000 7f173ffff640 1 -- 172.21.15.96:0/2047753875 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1750003ec0 con 0x7f175819cfc0 2024-04-25T21:02:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 Processor -- start 2024-04-25T21:02:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 -- start start 2024-04-25T21:02:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e6576ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576cf40 0x555e657772c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 0x555e65779bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f173ffff640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f17400a3dc0 0x7f17400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555e656d1970 con 0x555e65777800 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555e6566e4a0 con 0x555e6576cf40 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175fdb8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e656cd930 con 0x555e6576c620 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 0x555e65779bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e6576ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576cf40 0x555e657772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 0x555e65779bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40992/0 (socket says 172.21.15.96:40992) 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f17400a3dc0 0x7f17400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175759e640 1 -- 172.21.15.96:0/3618243783 learned_addr learned my addr 172.21.15.96:0/3618243783 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f173ffff640 1 -- 172.21.15.96:0/2047753875 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f17500d0c30 con 0x7f175819cfc0 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 -- 172.21.15.96:0/3618243783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576cf40 msgr2=0x555e657772c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576cf40 0x555e657772c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 -- 172.21.15.96:0/3618243783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 msgr2=0x555e65779bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 0x555e65779bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175659c640 1 -- 172.21.15.96:0/3618243783 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e653e4210 con 0x555e6576c620 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.001+0000 7f175759e640 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 0x555e65779bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f175659c640 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e6576ca00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1758069280 tx=0x7f174c010ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78a5c53733d82c8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f175659c640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f17400a3dc0 0x7f17400a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f174c00f850 tx=0x7f174c018000 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f173d7fa640 1 -- 172.21.15.96:0/3618243783 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c019040 con 0x555e6576c620 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f173d7fa640 1 -- 172.21.15.96:0/3618243783 <== mon.1 v2:172.21.15.139:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f174c00ed30 con 0x555e6576c620 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f173d7fa640 1 -- 172.21.15.96:0/3618243783 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c0047a0 con 0x555e6576c620 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f175fdb8740 1 -- 172.21.15.96:0/3618243783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 msgr2=0x555e6576ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.002+0000 7f175fdb8740 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e6576ca00 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1758069280 tx=0x7f174c010ee0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- 172.21.15.96:0/3618243783 shutdown_connections 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e65777800 0x555e65779bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576cf40 0x555e657772c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 --2- 172.21.15.96:0/3618243783 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e6576ca00 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- 172.21.15.96:0/3618243783 >> 172.21.15.96:0/3618243783 conn(0x555e65768b30 msgr2=0x555e65768f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- 172.21.15.96:0/3618243783 shutdown_connections 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- 172.21.15.96:0/3618243783 wait complete. 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- start start 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 Processor -- start 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e65777200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e657744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e65774b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555e6577a3d0 con 0x555e6576cf40 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555e6566c0b0 con 0x555e65777800 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555e6577a550 con 0x555e6576c620 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.003+0000 7f175fdb8740 10 client.0 _ll_register_callbacks cb 0x555e65659d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e65777200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.009 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f175759e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e65774b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e65777200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:37560/0 (socket says 172.21.15.96:37560) 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 -- 172.21.15.96:0/905373398 learned_addr learned my addr 172.21.15.96:0/905373398 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f175659c640 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e657744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 -- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 msgr2=0x555e65774b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e65774b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 -- 172.21.15.96:0/905373398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 msgr2=0x555e657744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e657744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 -- 172.21.15.96:0/905373398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e654b0620 con 0x555e6576c620 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f1756d9d640 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e65777200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1748000c00 tx=0x7f17480316d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5746ab24e402c7d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f173aff5640 10 client.0 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f173aff5640 1 -- 172.21.15.96:0/905373398 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748035040 con 0x555e6576c620 2024-04-25T21:02:11.010 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f174c00ed30 con 0x555e6576c620 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f173aff5640 1 -- 172.21.15.96:0/905373398 <== mon.1 v2:172.21.15.139:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f1748004250 con 0x555e6576c620 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f173aff5640 1 -- 172.21.15.96:0/905373398 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1748004450 con 0x555e6576c620 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f175fdb8740 10 client.25228 fetch_fsmap learned FSMap version 29 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.004+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e657863d0 con 0x555e6576c620 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f173aff5640 1 -- 172.21.15.96:0/905373398 <== mon.1 v2:172.21.15.139:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f174c00ed30 con 0x555e6576c620 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f173aff5640 1 -- 172.21.15.96:0/905373398 <== mon.1 v2:172.21.15.139:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f174804f020 con 0x555e6576c620 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175fdb8740 10 client.25228 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175fdb8740 10 client.25228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175fdb8740 20 client.25228 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175fdb8740 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e65787840 0x555e65789c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175fdb8740 4 client.25228 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e656df390 con 0x555e65787840 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175659c640 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e65787840 0x555e65789c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.011 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f173aff5640 10 client.25228 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:11.012 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.005+0000 7f175659c640 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e65787840 0x555e65789c30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd3c3269813664e3 server_cookie=e00c2bf6fe099df in_seq=0 out_seq=0 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f173aff5640 1 -- 172.21.15.96:0/905373398 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e656df390 con 0x555e65787840 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f173aff5640 10 client.25228 handle_command_reply: tid=0 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 client.25228 shutdown 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 msgr2=0x555e65777200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e65777200 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1748000c00 tx=0x7f17480316d0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 shutdown_connections 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e65787840 0x555e65789c30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e65774b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e657744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 --2- 172.21.15.96:0/905373398 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576c620 0x555e65777200 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f173aff5640 0 client.25228 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 >> 172.21.15.96:0/905373398 conn(0x555e65768b30 msgr2=0x555e6576e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:11.013 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 shutdown_connections 2024-04-25T21:02:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.006+0000 7f175fdb8740 1 -- 172.21.15.96:0/905373398 wait complete. 2024-04-25T21:02:11.014 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.007+0000 7f175fdb8740 20 client.25228 trim_cache size 0 max 16384 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.026+0000 7f175fdb8740 1 Processor -- start 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.026+0000 7f175fdb8740 1 -- start start 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.026+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 0x555e65768430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.026+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576cf40 0x555e65783220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.026+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e6578f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f175fdb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17581b02f0 con 0x555e6576c620 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f175fdb8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17581adb60 con 0x555e65777800 2024-04-25T21:02:11.028 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f175fdb8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17581ab9d0 con 0x555e6576cf40 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 0x555e65768430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 0x555e65768430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41000/0 (socket says 172.21.15.96:41000) 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 -- 172.21.15.96:0/1270003378 learned_addr learned my addr 172.21.15.96:0/1270003378 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f175659c640 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576cf40 0x555e65783220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f175759e640 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e6578f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 -- 172.21.15.96:0/1270003378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576cf40 msgr2=0x555e65783220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576cf40 0x555e65783220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 -- 172.21.15.96:0/1270003378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 msgr2=0x555e6578f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e6578f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 -- 172.21.15.96:0/1270003378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e65779f60 con 0x555e6576c620 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f1756d9d640 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 0x555e65768430 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1748009850 tx=0x7f17480045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8344d5eaab5f277a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f173cff9640 1 -- 172.21.15.96:0/1270003378 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748035030 con 0x555e6576c620 2024-04-25T21:02:11.029 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f173cff9640 1 -- 172.21.15.96:0/1270003378 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1748031740 con 0x555e6576c620 2024-04-25T21:02:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.027+0000 7f173cff9640 1 -- 172.21.15.96:0/1270003378 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1748004f00 con 0x555e6576c620 2024-04-25T21:02:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 -- 172.21.15.96:0/1270003378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 msgr2=0x555e65768430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 0x555e65768430 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1748009850 tx=0x7f17480045f0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 -- 172.21.15.96:0/1270003378 shutdown_connections 2024-04-25T21:02:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e65777800 0x555e6578f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.033 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e6576cf40 0x555e65783220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 --2- 172.21.15.96:0/1270003378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576c620 0x555e65768430 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 -- 172.21.15.96:0/1270003378 >> 172.21.15.96:0/1270003378 conn(0x555e65768b30 msgr2=0x555e65782b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 -- 172.21.15.96:0/1270003378 shutdown_connections 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.028+0000 7f175fdb8740 1 -- 172.21.15.96:0/1270003378 wait complete. 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 -- start start 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 Processor -- start 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576c620 0x555e6577d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e6577d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e65777800 0x555e6577de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17581ac020 con 0x555e6576cf40 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f17581ad490 con 0x555e6576c620 2024-04-25T21:02:11.034 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f17581ac6f0 con 0x555e65777800 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175fdb8740 10 client.0 _ll_register_callbacks cb 0x7f1748009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e6577d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e6577d8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:41008/0 (socket says 172.21.15.96:41008) 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 -- 172.21.15.96:0/2438786412 learned_addr learned my addr 172.21.15.96:0/2438786412 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f1756d9d640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576c620 0x555e6577d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175759e640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e65777800 0x555e6577de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 -- 172.21.15.96:0/2438786412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e65777800 msgr2=0x555e6577de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e65777800 0x555e6577de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 -- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576c620 msgr2=0x555e6577d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576c620 0x555e6577d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175659c640 1 -- 172.21.15.96:0/2438786412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e657ecf50 con 0x555e6576cf40 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f175759e640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e65777800 0x555e6577de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.029+0000 7f1756d9d640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576c620 0x555e6577d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f175659c640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e6577d8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e790 tx=0x7f174c005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e96623b23bce28ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f173aff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f173aff5640 1 -- 172.21.15.96:0/2438786412 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f174c005310 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f173aff5640 1 -- 172.21.15.96:0/2438786412 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f174c0054b0 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1748031740 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f173aff5640 1 -- 172.21.15.96:0/2438786412 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f174c041050 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f175fdb8740 10 client.15795 fetch_fsmap learned FSMap version 29 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e657ee220 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.030+0000 7f173aff5640 1 -- 172.21.15.96:0/2438786412 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1748031740 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f173aff5640 1 -- 172.21.15.96:0/2438786412 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8346+0+0 (secure 0 0 0) 0x7f174c0336d0 con 0x555e6576cf40 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f175fdb8740 10 client.15795 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f175fdb8740 10 client.15795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f175fdb8740 20 client.15795 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f175fdb8740 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e657ee770 0x555e657f0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f175fdb8740 4 client.15795 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:02:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e656d9200 con 0x555e657ee770 2024-04-25T21:02:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f1756d9d640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e657ee770 0x555e657f0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.031+0000 7f173aff5640 10 client.15795 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.032+0000 7f1756d9d640 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e657ee770 0x555e657f0b90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5445e452200b52be server_cookie=e352bec80d8f3920 in_seq=0 out_seq=0 2024-04-25T21:02:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f173aff5640 1 -- 172.21.15.96:0/2438786412 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e656d9200 con 0x555e657ee770 2024-04-25T21:02:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f173aff5640 10 client.15795 handle_command_reply: tid=0 2024-04-25T21:02:11.054 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 client.15795 shutdown 2024-04-25T21:02:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 msgr2=0x555e6577d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.055 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e6577d8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f174c00e790 tx=0x7f174c005d30 comp rx=0 tx=0).stop 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 shutdown_connections 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555e657ee770 0x555e657f0b90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555e65777800 0x555e6577de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555e6576cf40 0x555e6577d8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.054+0000 7f175fdb8740 1 --2- 172.21.15.96:0/2438786412 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555e6576c620 0x555e6577d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 >> 172.21.15.96:0/2438786412 conn(0x555e65768b30 msgr2=0x555e65767420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f173aff5640 0 client.15795 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 shutdown_connections 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175fdb8740 1 -- 172.21.15.96:0/2438786412 wait complete. 2024-04-25T21:02:11.056 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175fdb8740 20 client.15795 trim_cache size 0 max 16384 2024-04-25T21:02:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:02:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:02:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:02:11.057 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f17400a3dc0 msgr2=0x7f17400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175d0a0640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f17400a3dc0 0x7f17400a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f174c00f850 tx=0x7f174c018000 comp rx=0 tx=0).stop 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175819cfc0 msgr2=0x7f175819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.055+0000 7f175d0a0640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175819cfc0 0x7f175819d470 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f175000ec50 tx=0x7f175000c6a0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 shutdown_connections 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f17400a3dc0 0x7f17400a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f175819cfc0 0x7f175819d470 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1758107270 0x7f1758198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 --2- 172.21.15.96:0/2047753875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17581021a0 0x7f1758198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 >> 172.21.15.96:0/2047753875 conn(0x7f17580fda00 msgr2=0x7f17580ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 shutdown_connections 2024-04-25T21:02:11.058 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:11.056+0000 7f175d0a0640 1 -- 172.21.15.96:0/2047753875 wait complete. 2024-04-25T21:02:12.460 INFO:tasks.workunit.client.0.smithi096.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-04-25T21:02:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:12 smithi096 ceph-mon[29288]: pgmap v140: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 1.06k op/s 2024-04-25T21:02:12.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:12 smithi097 ceph-mon[32667]: pgmap v140: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 1.06k op/s 2024-04-25T21:02:12.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:12 smithi139 ceph-mon[33027]: pgmap v140: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 4.0 MiB/s wr, 1.06k op/s 2024-04-25T21:02:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:14 smithi096 ceph-mon[29288]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 771 op/s 2024-04-25T21:02:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:14 smithi097 ceph-mon[32667]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 771 op/s 2024-04-25T21:02:14.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:14 smithi139 ceph-mon[33027]: pgmap v141: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 771 op/s 2024-04-25T21:02:15.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:15 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:15.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:15 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:15.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:15 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:02:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:16 smithi096 ceph-mon[29288]: pgmap v142: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.24k op/s 2024-04-25T21:02:16.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:16 smithi097 ceph-mon[32667]: pgmap v142: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.24k op/s 2024-04-25T21:02:16.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:16 smithi139 ceph-mon[33027]: pgmap v142: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.24k op/s 2024-04-25T21:02:18.480 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x63e5d2a 2024-04-25T21:02:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:18 smithi096 ceph-mon[29288]: pgmap v143: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 934 op/s 2024-04-25T21:02:18.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:18 smithi097 ceph-mon[32667]: pgmap v143: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 934 op/s 2024-04-25T21:02:18.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:18 smithi139 ceph-mon[33027]: pgmap v143: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 934 op/s 2024-04-25T21:02:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:20 smithi096 ceph-mon[29288]: pgmap v144: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 934 op/s 2024-04-25T21:02:20.813 INFO:tasks.workunit.client.0.smithi096.stdout:truncating to largest ever: 0x63e9b52 2024-04-25T21:02:20.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:20 smithi097 ceph-mon[32667]: pgmap v144: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 934 op/s 2024-04-25T21:02:20.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:20 smithi139 ceph-mon[33027]: pgmap v144: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 2.5 MiB/s wr, 934 op/s 2024-04-25T21:02:22.618 INFO:tasks.workunit.client.0.smithi096.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-04-25T21:02:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:22 smithi096 ceph-mon[29288]: pgmap v145: 97 pgs: 97 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 3.7 MiB/s wr, 1.40k op/s 2024-04-25T21:02:22.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:22 smithi097 ceph-mon[32667]: pgmap v145: 97 pgs: 97 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 3.7 MiB/s wr, 1.40k op/s 2024-04-25T21:02:22.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:22 smithi139 ceph-mon[33027]: pgmap v145: 97 pgs: 97 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 3.7 MiB/s wr, 1.40k op/s 2024-04-25T21:02:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 -- 172.21.15.96:0/956586784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc106170 msgr2=0x7fcfdc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.163 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 --2- 172.21.15.96:0/956586784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc106170 0x7fcfdc10dd40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4009ca0 tx=0x7fcfd402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 -- 172.21.15.96:0/956586784 shutdown_connections 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 --2- 172.21.15.96:0/956586784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc106170 0x7fcfdc10dd40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 --2- 172.21.15.96:0/956586784 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc101aa0 0x7fcfdc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 --2- 172.21.15.96:0/956586784 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 -- 172.21.15.96:0/956586784 >> 172.21.15.96:0/956586784 conn(0x7fcfdc077fd0 msgr2=0x7fcfdc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 -- 172.21.15.96:0/956586784 shutdown_connections 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.160+0000 7fcfe2308640 1 -- 172.21.15.96:0/956586784 wait complete. 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 Processor -- start 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 -- start start 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfdc106170 0x7fcfdc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc19be10 0x7fcfdc19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.164 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc111110 con 0x7fcfdc19be10 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc110f90 con 0x7fcfdc1010f0 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe2308640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc111290 con 0x7fcfdc106170 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfe08a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc19be10 0x7fcfdc19c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc197550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:49216/0 (socket says 172.21.15.96:49216) 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.161+0000 7fcfdbfff640 1 -- 172.21.15.96:0/1332771332 learned_addr learned my addr 172.21.15.96:0/1332771332 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfdc106170 0x7fcfdc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdbfff640 1 -- 172.21.15.96:0/1332771332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfdc106170 msgr2=0x7fcfdc197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfdc106170 0x7fcfdc197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdbfff640 1 -- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc19be10 msgr2=0x7fcfdc19c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc19be10 0x7fcfdc19c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdbfff640 1 -- 172.21.15.96:0/1332771332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfdc19c900 con 0x7fcfdc1010f0 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc19be10 0x7fcfdc19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc197550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e9f0 tx=0x7fcfcc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:24.165 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfd97fa640 1 -- 172.21.15.96:0/1332771332 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00cd80 con 0x7fcfdc1010f0 2024-04-25T21:02:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfd97fa640 1 -- 172.21.15.96:0/1332771332 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfcc0108e0 con 0x7fcfdc1010f0 2024-04-25T21:02:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfdc19cbf0 con 0x7fcfdc1010f0 2024-04-25T21:02:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfd97fa640 1 -- 172.21.15.96:0/1332771332 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc018980 con 0x7fcfdc1010f0 2024-04-25T21:02:24.166 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.162+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfdc1a43e0 con 0x7fcfdc1010f0 2024-04-25T21:02:24.172 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.165+0000 7fcfd97fa640 1 -- 172.21.15.96:0/1332771332 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcfcc010430 con 0x7fcfdc1010f0 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.165+0000 7fcfe5026740 1 Processor -- start 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.165+0000 7fcfe5026740 1 -- start start 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.165+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910caf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.165+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cbc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.165+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cbee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe5026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555910c14970 con 0x555910caf480 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe5026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555910bb14a0 con 0x555910cafda0 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe5026740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555910c10930 con 0x555910cbca10 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfd97fa640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910caf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cbc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910caf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56388/0 (socket says 172.21.15.96:56388) 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 -- 172.21.15.96:0/1939853392 learned_addr learned my addr 172.21.15.96:0/1939853392 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.173 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfd97fa640 1 -- 172.21.15.96:0/1332771332 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fcfcc0d0b20 con 0x7fcfdc1010f0 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cbee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 -- 172.21.15.96:0/1939853392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 msgr2=0x555910cbee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cbee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 -- 172.21.15.96:0/1939853392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 msgr2=0x555910cbc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cbc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 -- 172.21.15.96:0/1939853392 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555910927210 con 0x555910caf480 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.166+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910caf860 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcfd402aeb0 tx=0x7fcfd4031550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7469b2383cc22d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.167+0000 7fcfc27fc640 1 -- 172.21.15.96:0/1939853392 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4004b40 con 0x555910caf480 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.167+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfd404f000 tx=0x7fcfd404d000 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.167+0000 7fcfc27fc640 1 -- 172.21.15.96:0/1939853392 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4050070 con 0x555910caf480 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.167+0000 7fcfc27fc640 1 -- 172.21.15.96:0/1939853392 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd402f990 con 0x555910caf480 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.167+0000 7fcfe5026740 1 -- 172.21.15.96:0/1939853392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 msgr2=0x555910caf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.167+0000 7fcfe5026740 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910caf860 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcfd402aeb0 tx=0x7fcfd4031550 comp rx=0 tx=0).stop 2024-04-25T21:02:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- 172.21.15.96:0/1939853392 shutdown_connections 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cbee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cbc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 --2- 172.21.15.96:0/1939853392 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910caf860 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- 172.21.15.96:0/1939853392 >> 172.21.15.96:0/1939853392 conn(0x555910cabb30 msgr2=0x555910cabf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- 172.21.15.96:0/1939853392 shutdown_connections 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- 172.21.15.96:0/1939853392 wait complete. 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- start start 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 Processor -- start 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cbc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 0x555910cb9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555910cbf4c0 con 0x555910cafda0 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x555910baf0b0 con 0x555910caf480 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.168+0000 7fcfe5026740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x555910cbf640 con 0x555910cbca10 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 0x555910cb9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 0x555910cb9680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56404/0 (socket says 172.21.15.96:56404) 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfe5026740 10 client.0 _ll_register_callbacks cb 0x555910b9cd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2819202354 learned_addr learned my addr 172.21.15.96:0/2819202354 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cbc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2819202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 msgr2=0x555910cb9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb9c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 msgr2=0x555910cbc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cbc390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2819202354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559109f3620 con 0x555910cafda0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cbc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 0x555910cb9680 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009fb0 tx=0x7fcfd00128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f87ba1bf1abfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2819202354 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd001f060 con 0x555910cafda0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4050070 con 0x555910cafda0 2024-04-25T21:02:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2819202354 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fcfd0010700 con 0x555910cafda0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2819202354 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfd0010900 con 0x555910cafda0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfe5026740 10 client.15807 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.169+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555910cb3ed0 con 0x555910cafda0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2819202354 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4050070 con 0x555910cafda0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2819202354 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fcfd0022020 con 0x555910cafda0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfe5026740 10 client.15807 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfe5026740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfe5026740 20 client.15807 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910ccede0 0x555910cd11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfe5026740 4 client.15807 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555910c22390 con 0x555910ccede0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910ccede0 0x555910cd11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910ccede0 0x555910cd11d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963b25abb1628f1 server_cookie=2adb8bcc3c007c3a in_seq=0 out_seq=0 2024-04-25T21:02:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.170+0000 7fcfbb7fe640 10 client.15807 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2819202354 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555910c22390 con 0x555910ccede0 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfbb7fe640 10 client.15807 handle_command_reply: tid=0 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 client.15807 shutdown 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 msgr2=0x555910cb9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 0x555910cb9680 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009fb0 tx=0x7fcfd00128e0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 shutdown_connections 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910ccede0 0x555910cd11d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cafda0 0x555910cb9680 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2819202354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cbc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 >> 172.21.15.96:0/2819202354 conn(0x555910cabb30 msgr2=0x555910cb1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfbb7fe640 0 client.15807 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 shutdown_connections 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 1 -- 172.21.15.96:0/2819202354 wait complete. 2024-04-25T21:02:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.171+0000 7fcfe5026740 20 client.15807 trim_cache size 0 max 16384 2024-04-25T21:02:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.191+0000 7fcfe5026740 1 Processor -- start 2024-04-25T21:02:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.191+0000 7fcfe5026740 1 -- start start 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cc83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 0x555910cbbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 0x555910cd6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe5026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1ae040 con 0x555910cbca10 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe5026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1ab8e0 con 0x555910caf480 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe5026740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1a9730 con 0x555910cafda0 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 0x555910cd6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cc83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 0x555910cbbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 0x555910cd6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56410/0 (socket says 172.21.15.96:56410) 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 0x555910cbbec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:58676/0 (socket says 172.21.15.96:58676) 2024-04-25T21:02:24.194 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cc83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:49228/0 (socket says 172.21.15.96:49228) 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/4160708706 learned_addr learned my addr 172.21.15.96:0/4160708706 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 -- 172.21.15.96:0/4160708706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 msgr2=0x555910cbbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 0x555910cbbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 -- 172.21.15.96:0/4160708706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 msgr2=0x555910cc83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cc83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfe08a5640 1 -- 172.21.15.96:0/4160708706 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555910cc8f30 con 0x555910cbca10 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 0x555910cbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.192+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cc83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.193+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 0x555910cd6940 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4004af0 tx=0x7fcfd402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f22ef6c109dbe158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.193+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/4160708706 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd402fbc0 con 0x555910cbca10 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.193+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/4160708706 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4058070 con 0x555910cbca10 2024-04-25T21:02:24.195 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.193+0000 7fcfc1ffb640 1 -- 172.21.15.96:0/4160708706 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4004040 con 0x555910cbca10 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 -- 172.21.15.96:0/4160708706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 msgr2=0x555910cd6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 0x555910cd6940 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fcfd4004af0 tx=0x7fcfd402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 -- 172.21.15.96:0/4160708706 shutdown_connections 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910cbca10 0x555910cd6940 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cafda0 0x555910cbbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 --2- 172.21.15.96:0/4160708706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910caf480 0x555910cc83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 -- 172.21.15.96:0/4160708706 >> 172.21.15.96:0/4160708706 conn(0x555910cabb30 msgr2=0x555910cb9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 -- 172.21.15.96:0/4160708706 shutdown_connections 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 -- 172.21.15.96:0/4160708706 wait complete. 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.194+0000 7fcfe5026740 1 -- start start 2024-04-25T21:02:24.199 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 Processor -- start 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910cc48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cc4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1a9da0 con 0x555910caf480 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1ab210 con 0x555910cafda0 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcfdc1aa470 con 0x555910cbca10 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe5026740 10 client.0 _ll_register_callbacks cb 0x7fcfd0009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910cc48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cc4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cc4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:49232/0 (socket says 172.21.15.96:49232) 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910cc48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:56412/0 (socket says 172.21.15.96:56412) 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2106408354 learned_addr learned my addr 172.21.15.96:0/2106408354 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfe08a5640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.200 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2106408354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 msgr2=0x555910cb5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2106408354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 msgr2=0x555910cc48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.195+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910cc48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfdb7fe640 1 -- 172.21.15.96:0/2106408354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555910cb5fa0 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910cc48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfdb7fe640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cc4e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0019dc0 tx=0x7fcfd0010cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fce4c329aa02bdaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfbb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2106408354 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0005250 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2106408354 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fcfd0010e40 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd4058070 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2106408354 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fcfd00053f0 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfe5026740 10 client.25286 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555910d34af0 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.196+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2106408354 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd4058070 con 0x555910cafda0 2024-04-25T21:02:24.201 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2106408354 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fcfd000e7e0 con 0x555910cafda0 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfe5026740 10 client.25286 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfe5026740 10 client.25286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfe5026740 20 client.25286 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910d352a0 0x555910d376c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfe5026740 4 client.25286 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555910c1c200 con 0x555910d352a0 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910d352a0 0x555910d376c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfbb7fe640 10 client.25286 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:24.202 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.197+0000 7fcfdbfff640 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910d352a0 0x555910d376c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5ec533e64eb814e server_cookie=5f465604885b582 in_seq=0 out_seq=0 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfbb7fe640 1 -- 172.21.15.96:0/2106408354 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555910c1c200 con 0x555910d352a0 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfbb7fe640 10 client.25286 handle_command_reply: tid=0 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 client.25286 shutdown 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 msgr2=0x555910cc4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cc4e10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0019dc0 tx=0x7fcfd0010cd0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 shutdown_connections 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x555910d352a0 0x555910d376c0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x555910cbca10 0x555910cb5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.212 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555910cafda0 0x555910cc4e10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 --2- 172.21.15.96:0/2106408354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555910caf480 0x555910cc48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfbb7fe640 0 client.25286 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 >> 172.21.15.96:0/2106408354 conn(0x555910cabb30 msgr2=0x555910cb4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 shutdown_connections 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 1 -- 172.21.15.96:0/2106408354 wait complete. 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.211+0000 7fcfe5026740 20 client.25286 trim_cache size 0 max 16384 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:02:24.213 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:02:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcfc40a3dc0 msgr2=0x7fcfc40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fcfd404f000 tx=0x7fcfd404d000 comp rx=0 tx=0).stop 2024-04-25T21:02:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 msgr2=0x7fcfdc197550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:24.214 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc197550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00e9f0 tx=0x7fcfcc00eeb0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 shutdown_connections 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcfc40a3dc0 0x7fcfc40a6280 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcfdc19be10 0x7fcfdc19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcfdc106170 0x7fcfdc197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 --2- 172.21.15.96:0/1332771332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcfdc1010f0 0x7fcfdc197550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 >> 172.21.15.96:0/1332771332 conn(0x7fcfdc077fd0 msgr2=0x7fcfdc0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 shutdown_connections 2024-04-25T21:02:24.215 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:24.212+0000 7fcfe2308640 1 -- 172.21.15.96:0/1332771332 wait complete. 2024-04-25T21:02:24.414 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x63ed6b9 2024-04-25T21:02:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:24 smithi096 ceph-mon[29288]: pgmap v146: 97 pgs: 97 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 927 op/s 2024-04-25T21:02:24.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:24 smithi097 ceph-mon[32667]: pgmap v146: 97 pgs: 97 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 927 op/s 2024-04-25T21:02:24.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:24 smithi139 ceph-mon[33027]: pgmap v146: 97 pgs: 97 active+clean; 61 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 2.6 MiB/s wr, 927 op/s 2024-04-25T21:02:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:26 smithi096 ceph-mon[29288]: pgmap v147: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:02:26.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:26 smithi097 ceph-mon[32667]: pgmap v147: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:02:26.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:26 smithi139 ceph-mon[33027]: pgmap v147: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 MiB/s rd, 3.8 MiB/s wr, 1.38k op/s 2024-04-25T21:02:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:28 smithi096 ceph-mon[29288]: pgmap v148: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 910 op/s 2024-04-25T21:02:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:28 smithi097 ceph-mon[32667]: pgmap v148: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 910 op/s 2024-04-25T21:02:28.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:28 smithi139 ceph-mon[33027]: pgmap v148: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 910 op/s 2024-04-25T21:02:30.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:30 smithi096 ceph-mon[29288]: pgmap v149: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 910 op/s 2024-04-25T21:02:30.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:30 smithi097 ceph-mon[32667]: pgmap v149: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 910 op/s 2024-04-25T21:02:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:30 smithi139 ceph-mon[33027]: pgmap v149: 97 pgs: 97 active+clean; 59 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 2.5 MiB/s wr, 910 op/s 2024-04-25T21:02:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:32 smithi096 ceph-mon[29288]: pgmap v150: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.7 MiB/s wr, 1.35k op/s 2024-04-25T21:02:32.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:32 smithi097 ceph-mon[32667]: pgmap v150: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.7 MiB/s wr, 1.35k op/s 2024-04-25T21:02:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:32 smithi139 ceph-mon[33027]: pgmap v150: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 3.7 MiB/s wr, 1.35k op/s 2024-04-25T21:02:34.033 INFO:tasks.workunit.client.0.smithi096.stdout:copying to largest ever: 0x63ef9ee 2024-04-25T21:02:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:34 smithi096 ceph-mon[29288]: pgmap v151: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 894 op/s 2024-04-25T21:02:34.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:34 smithi097 ceph-mon[32667]: pgmap v151: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 894 op/s 2024-04-25T21:02:34.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:34 smithi139 ceph-mon[33027]: pgmap v151: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 894 op/s 2024-04-25T21:02:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:36 smithi096 ceph-mon[29288]: pgmap v152: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.8 MiB/s wr, 1.33k op/s 2024-04-25T21:02:36.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:36 smithi097 ceph-mon[32667]: pgmap v152: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.8 MiB/s wr, 1.33k op/s 2024-04-25T21:02:36.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:36 smithi139 ceph-mon[33027]: pgmap v152: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.8 MiB/s wr, 1.33k op/s 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2337640260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb4106190 msgr2=0x7fdfb410dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2337640260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb4106190 0x7fdfb410dd60 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdfac009ca0 tx=0x7fdfac02a8c0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2337640260 shutdown_connections 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2337640260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb4106190 0x7fdfb410dd60 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2337640260 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4101a70 0x7fdfb4105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2337640260 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfb41010c0 0x7fdfb41014a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2337640260 >> 172.21.15.96:0/2337640260 conn(0x7fdfb4077f70 msgr2=0x7fdfb4078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2337640260 shutdown_connections 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.321+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2337640260 wait complete. 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 Processor -- start 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 -- start start 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 0x7fdfb41994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfb4106190 0x7fdfb4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4199fb0 0x7fdfb41a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4111100 con 0x7fdfb41010c0 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4110f80 con 0x7fdfb4199fb0 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfbc16f640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb4111280 con 0x7fdfb4106190 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 0x7fdfb41994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 0x7fdfb41994e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37232/0 (socket says 172.21.15.96:37232) 2024-04-25T21:02:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/2743041041 learned_addr learned my addr 172.21.15.96:0/2743041041 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4199fb0 0x7fdfb41a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.322+0000 7fdfb970a640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfb4106190 0x7fdfb4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfba70c640 1 -- 172.21.15.96:0/2743041041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfb4106190 msgr2=0x7fdfb4199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfb4106190 0x7fdfb4199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfba70c640 1 -- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 msgr2=0x7fdfb41994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 0x7fdfb41994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfba70c640 1 -- 172.21.15.96:0/2743041041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfb41a4d60 con 0x7fdfb4199fb0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 0x7fdfb41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4199fb0 0x7fdfb41a4560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02ade0 tx=0x7fdfac031660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfaaffd640 1 -- 172.21.15.96:0/2743041041 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003b70 con 0x7fdfb4199fb0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfaaffd640 1 -- 172.21.15.96:0/2743041041 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac031910 con 0x7fdfb4199fb0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfb41a4ff0 con 0x7fdfb4199fb0 2024-04-25T21:02:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfaaffd640 1 -- 172.21.15.96:0/2743041041 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac02f950 con 0x7fdfb4199fb0 2024-04-25T21:02:37.327 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.323+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfb41a5510 con 0x7fdfb4199fb0 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.325+0000 7fdfaaffd640 1 -- 172.21.15.96:0/2743041041 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdfac02faf0 con 0x7fdfb4199fb0 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 Processor -- start 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 -- start start 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfaaffd640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdfa00a3dc0 0x7fdfa00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea216a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea2212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea223bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfaaffd640 1 -- 172.21.15.96:0/2743041041 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fdfac0f18c0 con 0x7fdfb4199fb0 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5620ea17b970 con 0x5620ea221800 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5620ea1184a0 con 0x5620ea216620 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfbef66740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5620ea177930 con 0x5620ea216f40 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfb970a640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdfa00a3dc0 0x7fdfa00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea216a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfba70c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea223bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea216a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:46782/0 (socket says 172.21.15.96:46782) 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/3318190718 learned_addr learned my addr 172.21.15.96:0/3318190718 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.326+0000 7fdfb970a640 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea2212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfba70c640 1 -- 172.21.15.96:0/3318190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 msgr2=0x5620ea2212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfba70c640 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea2212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfba70c640 1 -- 172.21.15.96:0/3318190718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 msgr2=0x5620ea216a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfba70c640 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea216a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfba70c640 1 -- 172.21.15.96:0/3318190718 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620e9e8e210 con 0x5620ea221800 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea216a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfb970a640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdfa00a3dc0 0x7fdfa00a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfb406ad60 tx=0x7fdfa4011860 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdfba70c640 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea223bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0ef190 tx=0x7fdfac0bd520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee4c2985db6a7b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdf9ffff640 1 -- 172.21.15.96:0/3318190718 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac003d10 con 0x5620ea221800 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdf9ffff640 1 -- 172.21.15.96:0/3318190718 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac036040 con 0x5620ea221800 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.327+0000 7fdf9ffff640 1 -- 172.21.15.96:0/3318190718 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0ba050 con 0x5620ea221800 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 -- 172.21.15.96:0/3318190718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 msgr2=0x5620ea223bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea223bf0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0ef190 tx=0x7fdfac0bd520 comp rx=0 tx=0).stop 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 -- 172.21.15.96:0/3318190718 shutdown_connections 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea223bf0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea2212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3318190718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea216a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 -- 172.21.15.96:0/3318190718 >> 172.21.15.96:0/3318190718 conn(0x5620ea212b30 msgr2=0x5620ea212f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 -- 172.21.15.96:0/3318190718 shutdown_connections 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 -- 172.21.15.96:0/3318190718 wait complete. 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 -- start start 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.328+0000 7fdfbef66740 1 Processor -- start 2024-04-25T21:02:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea221130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea21e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea21e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5620ea17b970 con 0x5620ea216620 2024-04-25T21:02:37.335 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x5620ea1160b0 con 0x5620ea216f40 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x5620ea2241f0 con 0x5620ea221800 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfbef66740 10 client.0 _ll_register_callbacks cb 0x5620ea103d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea221130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea221130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37246/0 (socket says 172.21.15.96:37246) 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/2673356895 learned_addr learned my addr 172.21.15.96:0/2673356895 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea21e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfba70c640 1 -- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 msgr2=0x5620ea21e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea21e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfba70c640 1 -- 172.21.15.96:0/2673356895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 msgr2=0x5620ea221130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea221130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.329+0000 7fdfba70c640 1 -- 172.21.15.96:0/2673356895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620e9f5a620 con 0x5620ea221800 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.330+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac036040 con 0x5620ea221800 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.330+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea221130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:37.336 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.331+0000 7fdfba70c640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea21e960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0ef1c0 tx=0x7fdfac0ef5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f62936523ed04b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdf9d7fa640 10 client.25258 ms_handle_connect on v2:172.21.15.139:3300/0 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/2673356895 <== mon.1 v2:172.21.15.139:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0d6070 con 0x5620ea221800 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/2673356895 <== mon.1 v2:172.21.15.139:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fdfac0bdc40 con 0x5620ea221800 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/2673356895 <== mon.1 v2:172.21.15.139:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfac003d10 con 0x5620ea221800 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 10 client.25258 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5620ea234210 con 0x5620ea221800 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/2673356895 <== mon.1 v2:172.21.15.139:3300/0 4 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fdfac0c61c0 con 0x5620ea221800 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 10 client.25258 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 10 client.25258 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 20 client.25258 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea235560 0x5620ea237950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 4 client.25258 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.332+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620ea189390 con 0x5620ea235560 2024-04-25T21:02:37.337 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.333+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea235560 0x5620ea237950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.333+0000 7fdf9d7fa640 10 client.25258 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.333+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea235560 0x5620ea237950 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78ea7f0bf2cda1be server_cookie=2cc21c47449532bb in_seq=0 out_seq=0 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/2673356895 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620ea189390 con 0x5620ea235560 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdf9d7fa640 10 client.25258 handle_command_reply: tid=0 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 client.25258 shutdown 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 msgr2=0x5620ea21e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea21e960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0ef1c0 tx=0x7fdfac0ef5b0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 shutdown_connections 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea235560 0x5620ea237950 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.338 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea21e960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdf9d7fa640 0 client.25258 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea21e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 --2- 172.21.15.96:0/2673356895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea221130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 >> 172.21.15.96:0/2673356895 conn(0x5620ea212b30 msgr2=0x5620ea218860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 shutdown_connections 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 1 -- 172.21.15.96:0/2673356895 wait complete. 2024-04-25T21:02:37.339 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.334+0000 7fdfbef66740 20 client.25258 trim_cache size 0 max 16384 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 Processor -- start 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 -- start start 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea22cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea219af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea211b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ae180 con 0x5620ea221800 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41aba20 con 0x5620ea216620 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfbef66740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a98d0 con 0x5620ea216f40 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfb970a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea219af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea22cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfba70c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea211b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.358 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfb970a640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea219af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:37780/0 (socket says 172.21.15.96:37780) 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfb970a640 1 -- 172.21.15.96:0/3028626473 learned_addr learned my addr 172.21.15.96:0/3028626473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.355+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea22cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:46794/0 (socket says 172.21.15.96:46794) 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfba70c640 1 -- 172.21.15.96:0/3028626473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 msgr2=0x5620ea219af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfba70c640 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea219af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfba70c640 1 -- 172.21.15.96:0/3028626473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 msgr2=0x5620ea22cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfba70c640 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea22cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfba70c640 1 -- 172.21.15.96:0/3028626473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ea237f30 con 0x5620ea221800 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfb970a640 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea219af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea22cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdfba70c640 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea211b30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0ef240 tx=0x7fdfac049000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6544235b23cb796c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdf9f7fe640 1 -- 172.21.15.96:0/3028626473 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0d6070 con 0x5620ea221800 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdf9f7fe640 1 -- 172.21.15.96:0/3028626473 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac036040 con 0x5620ea221800 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.356+0000 7fdf9f7fe640 1 -- 172.21.15.96:0/3028626473 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfac0bdc40 con 0x5620ea221800 2024-04-25T21:02:37.359 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 -- 172.21.15.96:0/3028626473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 msgr2=0x5620ea211b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea211b30 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0ef240 tx=0x7fdfac049000 comp rx=0 tx=0).stop 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 -- 172.21.15.96:0/3028626473 shutdown_connections 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea221800 0x5620ea211b30 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea216f40 0x5620ea219af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 --2- 172.21.15.96:0/3028626473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216620 0x5620ea22cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 -- 172.21.15.96:0/3028626473 >> 172.21.15.96:0/3028626473 conn(0x5620ea212b30 msgr2=0x5620ea211420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 -- 172.21.15.96:0/3028626473 shutdown_connections 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 -- 172.21.15.96:0/3028626473 wait complete. 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 -- start start 2024-04-25T21:02:37.360 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.357+0000 7fdfbef66740 1 Processor -- start 2024-04-25T21:02:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea224990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea224ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea228db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41a9f20 con 0x5620ea216620 2024-04-25T21:02:37.363 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41ab350 con 0x5620ea216f40 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdfb41aa5b0 con 0x5620ea221800 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 10 client.0 _ll_register_callbacks cb 0x7fdfac0b9d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea224990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea224990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:37254/0 (socket says 172.21.15.96:37254) 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/919899229 learned_addr learned my addr 172.21.15.96:0/919899229 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb970a640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea224ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfba70c640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea228db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/919899229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 msgr2=0x5620ea228db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea228db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 msgr2=0x5620ea224ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea224ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 -- 172.21.15.96:0/919899229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ea2293f0 con 0x5620ea216620 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfb9f0b640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea224990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00097e0 tx=0x7fdfb0013e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ca29aea9c71037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:37.364 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdf9d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/919899229 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0005220 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/919899229 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fdfb00053c0 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.358+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfac036040 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/919899229 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdfb002e050 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 10 client.15831 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5620ea29b550 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/919899229 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfac036040 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/919899229 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7fdfb001f440 con 0x5620ea216620 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 10 client.15831 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 10 client.15831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 20 client.15831 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea29bc80 0x5620ea29e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 4 client.15831 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:02:37.365 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620ea183200 con 0x5620ea29bc80 2024-04-25T21:02:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfb970a640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea29bc80 0x5620ea29e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdf9d7fa640 10 client.15831 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:37.366 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.359+0000 7fdfb970a640 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea29bc80 0x5620ea29e0a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8839a1bdde6f108d server_cookie=34d42831d5e5edf1 in_seq=0 out_seq=0 2024-04-25T21:02:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdf9d7fa640 1 -- 172.21.15.96:0/919899229 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620ea183200 con 0x5620ea29bc80 2024-04-25T21:02:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdf9d7fa640 10 client.15831 handle_command_reply: tid=0 2024-04-25T21:02:37.378 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 client.15831 shutdown 2024-04-25T21:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 msgr2=0x5620ea224990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea224990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00097e0 tx=0x7fdfb0013e20 comp rx=0 tx=0).stop 2024-04-25T21:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 shutdown_connections 2024-04-25T21:02:37.379 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x5620ea29bc80 0x5620ea29e0a0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x5620ea221800 0x5620ea228db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5620ea216f40 0x5620ea224ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdf9d7fa640 0 client.15831 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 --2- 172.21.15.96:0/919899229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ea216620 0x5620ea224990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 >> 172.21.15.96:0/919899229 conn(0x5620ea212b30 msgr2=0x5620ea218860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 shutdown_connections 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.378+0000 7fdfbef66740 1 -- 172.21.15.96:0/919899229 wait complete. 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbef66740 20 client.15831 trim_cache size 0 max 16384 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:02:37.380 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:02:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:02:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:02:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdfa00a3dc0 msgr2=0x7fdfa00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdfa00a3dc0 0x7fdfa00a6280 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdfb406ad60 tx=0x7fdfa4011860 comp rx=0 tx=0).stop 2024-04-25T21:02:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4199fb0 msgr2=0x7fdfb41a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:37.381 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4199fb0 0x7fdfb41a4560 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02ade0 tx=0x7fdfac031660 comp rx=0 tx=0).stop 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 shutdown_connections 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdfa00a3dc0 0x7fdfa00a6280 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdfb4199fb0 0x7fdfb41a4560 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdfb4106190 0x7fdfb4199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 --2- 172.21.15.96:0/2743041041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdfb41010c0 0x7fdfb41994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 >> 172.21.15.96:0/2743041041 conn(0x7fdfb4077f70 msgr2=0x7fdfb40fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 shutdown_connections 2024-04-25T21:02:37.382 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:37.379+0000 7fdfbc16f640 1 -- 172.21.15.96:0/2743041041 wait complete. 2024-04-25T21:02:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:38 smithi096 ceph-mon[29288]: pgmap v153: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 880 op/s 2024-04-25T21:02:38.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:38 smithi097 ceph-mon[32667]: pgmap v153: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 880 op/s 2024-04-25T21:02:38.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:38 smithi139 ceph-mon[33027]: pgmap v153: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 880 op/s 2024-04-25T21:02:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:40 smithi096 ceph-mon[29288]: pgmap v154: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 880 op/s 2024-04-25T21:02:40.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:40 smithi097 ceph-mon[32667]: pgmap v154: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 880 op/s 2024-04-25T21:02:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:40 smithi139 ceph-mon[33027]: pgmap v154: 97 pgs: 97 active+clean; 76 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.5 MiB/s wr, 880 op/s 2024-04-25T21:02:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:42 smithi096 ceph-mon[29288]: pgmap v155: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.8 MiB/s wr, 1.32k op/s 2024-04-25T21:02:42.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:42 smithi097 ceph-mon[32667]: pgmap v155: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.8 MiB/s wr, 1.32k op/s 2024-04-25T21:02:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:42 smithi139 ceph-mon[33027]: pgmap v155: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 3.8 MiB/s wr, 1.32k op/s 2024-04-25T21:02:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:44 smithi096 ceph-mon[29288]: pgmap v156: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.5 MiB/s wr, 879 op/s 2024-04-25T21:02:44.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:44 smithi097 ceph-mon[32667]: pgmap v156: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.5 MiB/s wr, 879 op/s 2024-04-25T21:02:44.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:44 smithi139 ceph-mon[33027]: pgmap v156: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.5 MiB/s wr, 879 op/s 2024-04-25T21:02:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:46 smithi096 ceph-mon[29288]: pgmap v157: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.7 MiB/s wr, 1.31k op/s 2024-04-25T21:02:46.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:46 smithi097 ceph-mon[32667]: pgmap v157: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.7 MiB/s wr, 1.31k op/s 2024-04-25T21:02:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:46 smithi139 ceph-mon[33027]: pgmap v157: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 MiB/s rd, 3.7 MiB/s wr, 1.31k op/s 2024-04-25T21:02:47.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:47 smithi097 ceph-mon[32667]: pgmap v158: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 878 op/s 2024-04-25T21:02:47.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:47 smithi139 ceph-mon[33027]: pgmap v158: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 878 op/s 2024-04-25T21:02:47.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:47 smithi096 ceph-mon[29288]: pgmap v158: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 878 op/s 2024-04-25T21:02:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:48 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:48 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:48.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:48 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:02:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:49 smithi097 ceph-mon[32667]: pgmap v159: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 878 op/s 2024-04-25T21:02:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:49.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:02:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:49 smithi139 ceph-mon[33027]: pgmap v159: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 878 op/s 2024-04-25T21:02:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:49.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:02:49.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:49 smithi096 ceph-mon[29288]: pgmap v159: 97 pgs: 97 active+clean; 70 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 2.5 MiB/s wr, 878 op/s 2024-04-25T21:02:49.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:02:49.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:02:49.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 -- 172.21.15.96:0/3427295409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc106170 msgr2=0x7f41fc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 --2- 172.21.15.96:0/3427295409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc106170 0x7f41fc10dd40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41f4009be0 tx=0x7f41f402a6c0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 -- 172.21.15.96:0/3427295409 shutdown_connections 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 --2- 172.21.15.96:0/3427295409 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc106170 0x7f41fc10dd40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 --2- 172.21.15.96:0/3427295409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc101aa0 0x7f41fc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 --2- 172.21.15.96:0/3427295409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 -- 172.21.15.96:0/3427295409 >> 172.21.15.96:0/3427295409 conn(0x7f41fc077fd0 msgr2=0x7f41fc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 -- 172.21.15.96:0/3427295409 shutdown_connections 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.487+0000 7f4203291640 1 -- 172.21.15.96:0/3427295409 wait complete. 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 Processor -- start 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 -- start start 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc106170 0x7f41fc1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc19ad30 0x7f41fc19b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10ffd0 con 0x7f41fc106170 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc10fe50 con 0x7f41fc1010f0 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f4203291640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc110150 con 0x7f41fc19ad30 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f420082c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc106170 0x7f41fc1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f420102d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f420102d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:38770/0 (socket says 172.21.15.96:38770) 2024-04-25T21:02:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f420082c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc106170 0x7f41fc1969b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57664/0 (socket says 172.21.15.96:57664) 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f420182e640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc19ad30 0x7f41fc19b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.488+0000 7f420102d640 1 -- 172.21.15.96:0/3094359345 learned_addr learned my addr 172.21.15.96:0/3094359345 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420102d640 1 -- 172.21.15.96:0/3094359345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc19ad30 msgr2=0x7f41fc19b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420102d640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc19ad30 0x7f41fc19b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420102d640 1 -- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc106170 msgr2=0x7f41fc1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420102d640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc106170 0x7f41fc1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420102d640 1 -- 172.21.15.96:0/3094359345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc19b820 con 0x7f41fc1010f0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420182e640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc19ad30 0x7f41fc19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f420102d640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc196470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41f800e9f0 tx=0x7f41f800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f41f27fc640 1 -- 172.21.15.96:0/3094359345 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f800cd80 con 0x7f41fc1010f0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f41f27fc640 1 -- 172.21.15.96:0/3094359345 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f80054d0 con 0x7f41fc1010f0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc19bb10 con 0x7f41fc1010f0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f41f27fc640 1 -- 172.21.15.96:0/3094359345 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f8010690 con 0x7f41fc1010f0 2024-04-25T21:02:50.492 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.489+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc1a3220 con 0x7f41fc1010f0 2024-04-25T21:02:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f41f27fc640 1 -- 172.21.15.96:0/3094359345 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f41f8004e90 con 0x7f41fc1010f0 2024-04-25T21:02:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 Processor -- start 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 -- start start 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2e6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56206f24b970 con 0x56206f2f1800 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56206f1e84a0 con 0x56206f2e6620 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f4205fab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56206f247930 con 0x56206f2e6f40 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f41f27fc640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f41e80a5fd0 0x7f41e80a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f420182e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2e6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f420182e640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2e6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:38786/0 (socket says 172.21.15.96:38786) 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f420182e640 1 -- 172.21.15.96:0/1192723856 learned_addr learned my addr 172.21.15.96:0/1192723856 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f420082c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f420182e640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f41e80a5fd0 0x7f41e80a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.492+0000 7f41f27fc640 1 -- 172.21.15.96:0/3094359345 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f41f80d0ba0 con 0x7f41fc1010f0 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420102d640 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420082c640 1 -- 172.21.15.96:0/1192723856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 msgr2=0x56206f2f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420082c640 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420082c640 1 -- 172.21.15.96:0/1192723856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 msgr2=0x56206f2e6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420082c640 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2e6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420082c640 1 -- 172.21.15.96:0/1192723856 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206ef5e210 con 0x56206f2f1800 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420182e640 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420082c640 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2f3bf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009530 tx=0x7f41ec011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a05254f27aae1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f420182e640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f41e80a5fd0 0x7f41e80a8490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005c80 tx=0x7f41f4004a40 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f41e77fe640 1 -- 172.21.15.96:0/1192723856 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec009db0 con 0x56206f2f1800 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f41e77fe640 1 -- 172.21.15.96:0/1192723856 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec0035f0 con 0x56206f2f1800 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.493+0000 7f41e77fe640 1 -- 172.21.15.96:0/1192723856 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec019610 con 0x56206f2f1800 2024-04-25T21:02:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 -- 172.21.15.96:0/1192723856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 msgr2=0x56206f2f3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2f3bf0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009530 tx=0x7f41ec011810 comp rx=0 tx=0).stop 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 -- 172.21.15.96:0/1192723856 shutdown_connections 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2f3bf0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 --2- 172.21.15.96:0/1192723856 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2e6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 -- 172.21.15.96:0/1192723856 >> 172.21.15.96:0/1192723856 conn(0x56206f2e2b30 msgr2=0x56206f2e2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 -- 172.21.15.96:0/1192723856 shutdown_connections 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 -- 172.21.15.96:0/1192723856 wait complete. 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.494+0000 7f4205fab740 1 -- start start 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 Processor -- start 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 0x56206f2f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2ee410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2f1800 0x56206f2ee950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56206f2f4430 con 0x56206f2e6620 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x56206f1e60b0 con 0x56206f2e6f40 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x56206f2f45b0 con 0x56206f2f1800 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f4205fab740 10 client.0 _ll_register_callbacks cb 0x56206f1d3d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 0x56206f2f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 0x56206f2f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57692/0 (socket says 172.21.15.96:57692) 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 -- 172.21.15.96:0/2128954671 learned_addr learned my addr 172.21.15.96:0/2128954671 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420082c640 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 -- 172.21.15.96:0/2128954671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2f1800 msgr2=0x56206f2ee950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2f1800 0x56206f2ee950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 -- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 msgr2=0x56206f2ee410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.495+0000 7f420102d640 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2ee410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f420102d640 1 -- 172.21.15.96:0/2128954671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206f02a620 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f420102d640 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 0x56206f2f1120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f41f809cb20 tx=0x7f41f809c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e72d972327b747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f41e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f41e4ff9640 1 -- 172.21.15.96:0/2128954671 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f809d290 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f41e4ff9640 1 -- 172.21.15.96:0/2128954671 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f41f80bd0b0 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41ec0035f0 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f41e4ff9640 1 -- 172.21.15.96:0/2128954671 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41f809cb90 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 10 client.15843 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56206f3003b0 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f41e4ff9640 1 -- 172.21.15.96:0/2128954671 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ec0035f0 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f41e4ff9640 1 -- 172.21.15.96:0/2128954671 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f41f80ce650 con 0x56206f2e6620 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 10 client.15843 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:50.504 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 20 client.15843 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.496+0000 7f4205fab740 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f303cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.497+0000 7f4205fab740 4 client.15843 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.497+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56206f259390 con 0x56206f301870 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.497+0000 7f420082c640 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f303cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.497+0000 7f41e4ff9640 10 client.15843 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.497+0000 7f420082c640 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f303cc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d10848a9a9972466 server_cookie=9738440501149f1a in_seq=0 out_seq=0 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f41e4ff9640 1 -- 172.21.15.96:0/2128954671 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56206f259390 con 0x56206f301870 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f41e4ff9640 10 client.15843 handle_command_reply: tid=0 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 client.15843 shutdown 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 msgr2=0x56206f2f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 0x56206f2f1120 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f41f809cb20 tx=0x7f41f809c6d0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 shutdown_connections 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f303cc0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2f1800 0x56206f2ee950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.505 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f41e4ff9640 0 client.15843 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2ee410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 --2- 172.21.15.96:0/2128954671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2e6620 0x56206f2f1120 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 >> 172.21.15.96:0/2128954671 conn(0x56206f2e2b30 msgr2=0x56206f2e8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 shutdown_connections 2024-04-25T21:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 1 -- 172.21.15.96:0/2128954671 wait complete. 2024-04-25T21:02:50.506 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.498+0000 7f4205fab740 20 client.15843 trim_cache size 0 max 16384 2024-04-25T21:02:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 Processor -- start 2024-04-25T21:02:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 -- start start 2024-04-25T21:02:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2fd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.523 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2e1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1ae020 con 0x56206f2f1800 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1ab8c0 con 0x56206f2e6620 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f4205fab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1a9710 con 0x56206f2e6f40 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f420182e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2e1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f420182e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2e1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:57696/0 (socket says 172.21.15.96:57696) 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f420182e640 1 -- 172.21.15.96:0/1258305235 learned_addr learned my addr 172.21.15.96:0/1258305235 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f420102d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2fd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.522+0000 7f420082c640 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f420102d640 1 -- 172.21.15.96:0/1258305235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 msgr2=0x56206f2f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f420102d640 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f420102d640 1 -- 172.21.15.96:0/1258305235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 msgr2=0x56206f2e1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.524 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f420102d640 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2e1b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f420102d640 1 -- 172.21.15.96:0/1258305235 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56206f2ed3c0 con 0x56206f2e6620 2024-04-25T21:02:50.525 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f420102d640 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2fd2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f809c890 tx=0x7f41f809c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b177943efd6fda70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:50.528 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f41e6ffd640 1 -- 172.21.15.96:0/1258305235 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f809c4f0 con 0x56206f2e6620 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f41e6ffd640 1 -- 172.21.15.96:0/1258305235 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f809d290 con 0x56206f2e6620 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.523+0000 7f41e6ffd640 1 -- 172.21.15.96:0/1258305235 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f80ce5c0 con 0x56206f2e6620 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 -- 172.21.15.96:0/1258305235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 msgr2=0x56206f2fd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2fd2f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f41f809c890 tx=0x7f41f809c800 comp rx=0 tx=0).stop 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 -- 172.21.15.96:0/1258305235 shutdown_connections 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2e1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6f40 0x56206f2f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 --2- 172.21.15.96:0/1258305235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6620 0x56206f2fd2f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 -- 172.21.15.96:0/1258305235 >> 172.21.15.96:0/1258305235 conn(0x56206f2e2b30 msgr2=0x56206f2e1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 -- 172.21.15.96:0/1258305235 shutdown_connections 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.524+0000 7f4205fab740 1 -- 172.21.15.96:0/1258305235 wait complete. 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 -- start start 2024-04-25T21:02:50.529 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 Processor -- start 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 0x56206f2eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2eb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1a9d80 con 0x56206f2f1800 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1ab1f0 con 0x56206f2e6f40 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.525+0000 7f4205fab740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1aa450 con 0x56206f2e6620 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f4205fab740 10 client.0 _ll_register_callbacks cb 0x7f41f8098f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2eb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420102d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 0x56206f2eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420102d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 0x56206f2eb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.139:3300/0 says I am v2:172.21.15.96:46264/0 (socket says 172.21.15.96:46264) 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2eb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:38812/0 (socket says 172.21.15.96:38812) 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420102d640 1 -- 172.21.15.96:0/1891314245 learned_addr learned my addr 172.21.15.96:0/1891314245 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:02:50.530 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420182e640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 -- 172.21.15.96:0/1891314245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 msgr2=0x56206f2eb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 0x56206f2eb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 -- 172.21.15.96:0/1891314245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 msgr2=0x56206f2ebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2ebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 -- 172.21.15.96:0/1891314245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56206f2f3f60 con 0x56206f2e6f40 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420182e640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420102d640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 0x56206f2eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f420082c640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2eb640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41ec01b040 tx=0x7f41ec005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52542e54e033bc8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.526+0000 7f41e4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f41e4ff9640 1 -- 172.21.15.96:0/1891314245 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec01d040 con 0x56206f2e6f40 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f41e4ff9640 1 -- 172.21.15.96:0/1891314245 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f41ec004240 con 0x56206f2e6f40 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41f809d290 con 0x56206f2e6f40 2024-04-25T21:02:50.531 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f41e4ff9640 1 -- 172.21.15.96:0/1891314245 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f41ec03b050 con 0x56206f2e6f40 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 10 client.25319 fetch_fsmap learned FSMap version 30 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56206f367f70 con 0x56206f2e6f40 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f41e4ff9640 1 -- 172.21.15.96:0/1891314245 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f809d290 con 0x56206f2e6f40 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f41e4ff9640 1 -- 172.21.15.96:0/1891314245 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) ==== 8342+0+0 (secure 0 0 0) 0x7f41ec0037f0 con 0x56206f2e6f40 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 10 client.25319 fetch_fsmap finished waiting for FSMap version 30 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 10 client.25319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 20 client.25319 populate_metadata read hostname 'smithi096' 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f36a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 4 client.25319 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.527+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56206f253200 con 0x56206f301870 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.528+0000 7f420102d640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f36a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.528+0000 7f41e4ff9640 10 client.25319 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:50.532 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.528+0000 7f420102d640 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f36a640 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae1a8bb1692af596 server_cookie=40be12defa9a98d2 in_seq=0 out_seq=0 2024-04-25T21:02:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f41e4ff9640 1 -- 172.21.15.96:0/1891314245 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56206f253200 con 0x56206f301870 2024-04-25T21:02:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f41e4ff9640 10 client.25319 handle_command_reply: tid=0 2024-04-25T21:02:50.543 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 client.25319 shutdown 2024-04-25T21:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 msgr2=0x56206f2eb640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2eb640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f41ec01b040 tx=0x7f41ec005c20 comp rx=0 tx=0).stop 2024-04-25T21:02:50.544 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 shutdown_connections 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x56206f301870 0x56206f36a640 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56206f2f1800 0x56206f2ebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56206f2e6f40 0x56206f2eb640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 --2- 172.21.15.96:0/1891314245 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x56206f2e6620 0x56206f2eb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 >> 172.21.15.96:0/1891314245 conn(0x56206f2e2b30 msgr2=0x56206f2e1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f41e4ff9640 0 client.25319 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 shutdown_connections 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 1 -- 172.21.15.96:0/1891314245 wait complete. 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.542+0000 7f4205fab740 20 client.25319 trim_cache size 0 max 16384 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:02:50.545 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:02:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:02:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:02:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f41e80a5fd0 msgr2=0x7f41e80a8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f41e80a5fd0 0x7f41e80a8490 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f41f4005c80 tx=0x7f41f4004a40 comp rx=0 tx=0).stop 2024-04-25T21:02:50.546 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 msgr2=0x7f41fc196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc196470 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f41f800e9f0 tx=0x7f41f800edf0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 shutdown_connections 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f41e80a5fd0 0x7f41e80a8490 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f41fc19ad30 0x7f41fc19b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41fc106170 0x7f41fc1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 --2- 172.21.15.96:0/3094359345 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f41fc1010f0 0x7f41fc196470 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 >> 172.21.15.96:0/3094359345 conn(0x7f41fc077fd0 msgr2=0x7f41fc0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 shutdown_connections 2024-04-25T21:02:50.547 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:02:50.543+0000 7f4203291640 1 -- 172.21.15.96:0/3094359345 wait complete. 2024-04-25T21:02:51.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:51 smithi096 ceph-mon[29288]: pgmap v160: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.29k op/s 2024-04-25T21:02:52.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:51 smithi097 ceph-mon[32667]: pgmap v160: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.29k op/s 2024-04-25T21:02:52.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:51 smithi139 ceph-mon[33027]: pgmap v160: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.6 MiB/s wr, 1.29k op/s 2024-04-25T21:02:53.914 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:53 smithi097 ceph-mon[32667]: pgmap v161: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 846 op/s 2024-04-25T21:02:53.923 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:53 smithi139 ceph-mon[33027]: pgmap v161: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 846 op/s 2024-04-25T21:02:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:53 smithi096 ceph-mon[29288]: pgmap v161: 97 pgs: 97 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.3 MiB/s wr, 846 op/s 2024-04-25T21:02:55.416 INFO:tasks.workunit.client.0.smithi096.stdout:All 50000 operations completed A-OK! 2024-04-25T21:02:55.417 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-25T21:02:55.417 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-25T21:02:55.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:55 smithi096 ceph-mon[29288]: pgmap v162: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.4 MiB/s wr, 1.26k op/s 2024-04-25T21:02:56.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:55 smithi097 ceph-mon[32667]: pgmap v162: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.4 MiB/s wr, 1.26k op/s 2024-04-25T21:02:56.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:55 smithi139 ceph-mon[33027]: pgmap v162: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 24 MiB/s rd, 3.4 MiB/s wr, 1.26k op/s 2024-04-25T21:02:57.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:57 smithi096 ceph-mon[29288]: pgmap v163: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 829 op/s 2024-04-25T21:02:58.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:57 smithi097 ceph-mon[32667]: pgmap v163: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 829 op/s 2024-04-25T21:02:58.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:57 smithi139 ceph-mon[33027]: pgmap v163: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 829 op/s 2024-04-25T21:02:59.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:59 smithi096 ceph-mon[29288]: pgmap v164: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 829 op/s 2024-04-25T21:02:59.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:02:59 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:00.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:59 smithi097 ceph-mon[32667]: pgmap v164: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 829 op/s 2024-04-25T21:03:00.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:02:59 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:00.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:59 smithi139 ceph-mon[33027]: pgmap v164: 97 pgs: 97 active+clean; 83 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.3 MiB/s wr, 829 op/s 2024-04-25T21:03:00.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:02:59 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:01.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:01 smithi096 ceph-mon[29288]: pgmap v165: 97 pgs: 97 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.2 MiB/s wr, 958 op/s 2024-04-25T21:03:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:01 smithi097 ceph-mon[32667]: pgmap v165: 97 pgs: 97 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.2 MiB/s wr, 958 op/s 2024-04-25T21:03:02.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:01 smithi139 ceph-mon[33027]: pgmap v165: 97 pgs: 97 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 3.2 MiB/s wr, 958 op/s 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 -- 172.21.15.96:0/3605773614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 msgr2=0x7f330810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 --2- 172.21.15.96:0/3605773614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f330810dd40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f3300009ca0 tx=0x7f330002a8c0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 -- 172.21.15.96:0/3605773614 shutdown_connections 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 --2- 172.21.15.96:0/3605773614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f330810dd40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 --2- 172.21.15.96:0/3605773614 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3308101aa0 0x7f3308105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 --2- 172.21.15.96:0/3605773614 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33081010f0 0x7f33081014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.638 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 -- 172.21.15.96:0/3605773614 >> 172.21.15.96:0/3605773614 conn(0x7f3308077fd0 msgr2=0x7f33080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 -- 172.21.15.96:0/3605773614 shutdown_connections 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.635+0000 7f330d031640 1 -- 172.21.15.96:0/3605773614 wait complete. 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 Processor -- start 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 -- start start 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33081010f0 0x7f3308199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f3308199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3308199fd0 0x7f33081a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33081110c0 con 0x7f3308106170 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f3308110f40 con 0x7f33081010f0 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330d031640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f3308111240 con 0x7f3308199fd0 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f32fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f3308199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f32fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f3308199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39238/0 (socket says 172.21.15.96:39238) 2024-04-25T21:03:03.639 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f32fffff640 1 -- 172.21.15.96:0/2318384945 learned_addr learned my addr 172.21.15.96:0/2318384945 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f3306d9d640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33081010f0 0x7f3308199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.636+0000 7f330759e640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3308199fd0 0x7f33081a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f32fffff640 1 -- 172.21.15.96:0/2318384945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3308199fd0 msgr2=0x7f33081a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f32fffff640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3308199fd0 0x7f33081a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f32fffff640 1 -- 172.21.15.96:0/2318384945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33081010f0 msgr2=0x7f3308199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f32fffff640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33081010f0 0x7f3308199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f32fffff640 1 -- 172.21.15.96:0/2318384945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33081a4d50 con 0x7f3308106170 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f32fffff640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f3308199a40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d770 tx=0x7f32f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f3304d99640 1 -- 172.21.15.96:0/2318384945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0017070 con 0x7f3308106170 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f3304d99640 1 -- 172.21.15.96:0/2318384945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f0009c30 con 0x7f3308106170 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33081a4fe0 con 0x7f3308106170 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f3304d99640 1 -- 172.21.15.96:0/2318384945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f000f620 con 0x7f3308106170 2024-04-25T21:03:03.640 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.637+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33081a5500 con 0x7f3308106170 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.639+0000 7f3304d99640 1 -- 172.21.15.96:0/2318384945 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f32f001a070 con 0x7f3308106170 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.639+0000 7f330fe29740 1 Processor -- start 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 -- start start 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3304d99640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f32f40a3dc0 0x7f32f40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330759e640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f32f40a3dc0 0x7f32f40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3304d99640 1 -- 172.21.15.96:0/2318384945 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f32f00d0220 con 0x7f3308106170 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bc2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bcd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bd0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b54b23970 con 0x561b54bc2a30 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b54ac04a0 con 0x561b54bc3350 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330fe29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561b54b1f930 con 0x561b54bcdc10 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bc2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bc2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39244/0 (socket says 172.21.15.96:39244) 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 -- 172.21.15.96:0/2740012102 learned_addr learned my addr 172.21.15.96:0/2740012102 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f32fffff640 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bcd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330759e640 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bd0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 -- 172.21.15.96:0/2740012102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 msgr2=0x561b54bd0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bd0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 -- 172.21.15.96:0/2740012102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 msgr2=0x561b54bcd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bcd6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f3306d9d640 1 -- 172.21.15.96:0/2740012102 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b54836210 con 0x561b54bc2a30 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330759e640 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bd0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.640+0000 7f330759e640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f32f40a3dc0 0x7f32f40a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3300006010 tx=0x7f3300009800 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:03.648 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f3306d9d640 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bc2e10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009cf0 tx=0x7f32f8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9f4215c2db1082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f32fd7fa640 1 -- 172.21.15.96:0/2740012102 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8004230 con 0x561b54bc2a30 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f32fd7fa640 1 -- 172.21.15.96:0/2740012102 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f80043d0 con 0x561b54bc2a30 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f32fd7fa640 1 -- 172.21.15.96:0/2740012102 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8019840 con 0x561b54bc2a30 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f330fe29740 1 -- 172.21.15.96:0/2740012102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 msgr2=0x561b54bc2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f330fe29740 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bc2e10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f32f8009cf0 tx=0x7f32f8011810 comp rx=0 tx=0).stop 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.641+0000 7f330fe29740 1 -- 172.21.15.96:0/2740012102 shutdown_connections 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bd0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bcd6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 --2- 172.21.15.96:0/2740012102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bc2e10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- 172.21.15.96:0/2740012102 >> 172.21.15.96:0/2740012102 conn(0x561b54bbef40 msgr2=0x561b54bbf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- 172.21.15.96:0/2740012102 shutdown_connections 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- 172.21.15.96:0/2740012102 wait complete. 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- start start 2024-04-25T21:03:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 Processor -- start 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc2a30 0x561b54bcd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bcaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561b54bd0840 con 0x561b54bc3350 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x561b54abe0b0 con 0x561b54bc2a30 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x561b54bd09c0 con 0x561b54bcdc10 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330fe29740 10 client.0 _ll_register_callbacks cb 0x561b54aabd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f32fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc2a30 0x561b54bcd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f32fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bca760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39252/0 (socket says 172.21.15.96:39252) 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f32fffff640 1 -- 172.21.15.96:0/3471124532 learned_addr learned my addr 172.21.15.96:0/3471124532 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.642+0000 7f330759e640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bcaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f3306d9d640 1 -- 172.21.15.96:0/3471124532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 msgr2=0x561b54bcaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f3306d9d640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bcaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f3306d9d640 1 -- 172.21.15.96:0/3471124532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 msgr2=0x561b54bca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f3306d9d640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bca760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f3306d9d640 1 -- 172.21.15.96:0/3471124532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b54902620 con 0x561b54bc2a30 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f330759e640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bcaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f32fffff640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f3306d9d640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc2a30 0x561b54bcd470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32f8003f50 tx=0x7f32f80038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=adaa9b4480c7aea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f32ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3471124532 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8027070 con 0x561b54bc2a30 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f80043d0 con 0x561b54bc2a30 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3471124532 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f32f8005240 con 0x561b54bc2a30 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3471124532 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32f8033050 con 0x561b54bc2a30 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f330fe29740 10 client.25331 fetch_fsmap learned FSMap version 31 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561b54bdc5d0 con 0x561b54bc2a30 2024-04-25T21:03:03.651 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.643+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3471124532 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f80043d0 con 0x561b54bc2a30 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3471124532 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f32f8004b70 con 0x561b54bc2a30 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f330fe29740 10 client.25331 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f330fe29740 10 client.25331 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f330fe29740 20 client.25331 populate_metadata read hostname 'smithi096' 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f330fe29740 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54bdfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f330fe29740 4 client.25331 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b54b31390 con 0x561b54bdda40 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f32fffff640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54bdfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f32ee7fc640 10 client.25331 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.644+0000 7f32fffff640 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54bdfe30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c5090318914ba28 server_cookie=b533cb9be22c7627 in_seq=0 out_seq=0 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3471124532 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561b54b31390 con 0x561b54bdda40 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f32ee7fc640 10 client.25331 handle_command_reply: tid=0 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 client.25331 shutdown 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc2a30 msgr2=0x561b54bcd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.652 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc2a30 0x561b54bcd470 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32f8003f50 tx=0x7f32f80038a0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 shutdown_connections 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54bdfe30 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54bcaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f32ee7fc640 0 client.25331 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 --2- 172.21.15.96:0/3471124532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc2a30 0x561b54bcd470 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 >> 172.21.15.96:0/3471124532 conn(0x561b54bbef40 msgr2=0x561b54bb9cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 shutdown_connections 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 1 -- 172.21.15.96:0/3471124532 wait complete. 2024-04-25T21:03:03.653 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.645+0000 7f330fe29740 20 client.25331 trim_cache size 0 max 16384 2024-04-25T21:03:03.660 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.660+0000 7f330fe29740 1 Processor -- start 2024-04-25T21:03:03.661 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.660+0000 7f330fe29740 1 -- start start 2024-04-25T21:03:03.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330fe29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bc2a30 0x561b54bd9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330fe29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bc5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330fe29740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bcdc10 0x561b54bccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.665 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330fe29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33081ae1b0 con 0x561b54bc3350 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330fe29740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33081aba20 con 0x561b54bcdc10 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330fe29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33081a9890 con 0x561b54bc2a30 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bc5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bc5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39260/0 (socket says 172.21.15.96:39260) 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 -- 172.21.15.96:0/148789224 learned_addr learned my addr 172.21.15.96:0/148789224 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330759e640 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bcdc10 0x561b54bccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 -- 172.21.15.96:0/148789224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bc2a30 msgr2=0x561b54bd9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bc2a30 0x561b54bd9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 -- 172.21.15.96:0/148789224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bcdc10 msgr2=0x561b54bccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bcdc10 0x561b54bccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 -- 172.21.15.96:0/148789224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b54bbed70 con 0x561b54bc3350 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f330759e640 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bcdc10 0x561b54bccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:03:03.666 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fffff640 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bc5d80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009610 tx=0x7f32f0010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0cd90d58cb02a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fcff9640 1 -- 172.21.15.96:0/148789224 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f00ba070 con 0x561b54bc3350 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fcff9640 1 -- 172.21.15.96:0/148789224 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f00a4ea0 con 0x561b54bc3350 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.661+0000 7f32fcff9640 1 -- 172.21.15.96:0/148789224 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f0021040 con 0x561b54bc3350 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 -- 172.21.15.96:0/148789224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 msgr2=0x561b54bc5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bc5d80 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f32f0009610 tx=0x7f32f0010c00 comp rx=0 tx=0).stop 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 -- 172.21.15.96:0/148789224 shutdown_connections 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bcdc10 0x561b54bccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc3350 0x561b54bc5d80 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 --2- 172.21.15.96:0/148789224 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bc2a30 0x561b54bd9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 -- 172.21.15.96:0/148789224 >> 172.21.15.96:0/148789224 conn(0x561b54bbef40 msgr2=0x561b54bc5670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 -- 172.21.15.96:0/148789224 shutdown_connections 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 -- 172.21.15.96:0/148789224 wait complete. 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 -- start start 2024-04-25T21:03:03.667 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.662+0000 7f330fe29740 1 Processor -- start 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bd0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bd1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54c3bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33081a9ee0 con 0x561b54bc2a30 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f33081ab350 con 0x561b54bc3350 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f33081aa5b0 con 0x561b54bcdc10 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 10 client.0 _ll_register_callbacks cb 0x7f32f8005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bd0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bd0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39272/0 (socket says 172.21.15.96:39272) 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 -- 172.21.15.96:0/3783098174 learned_addr learned my addr 172.21.15.96:0/3783098174 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f32fffff640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bd1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 -- 172.21.15.96:0/3783098174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 msgr2=0x561b54c3bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330759e640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54c3bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.668 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54c3bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 -- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 msgr2=0x561b54bd1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bd1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 -- 172.21.15.96:0/3783098174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b54c3c460 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f3306d9d640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bd0ee0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f32f8003a30 tx=0x7f32f8019970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c77b7f86dbf80d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f32ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3783098174 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32f8027070 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3783098174 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7f32f8019d30 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32f00a4ea0 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3783098174 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f32f8009ea0 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 10 client.15867 fetch_fsmap learned FSMap version 31 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.663+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561b54c43820 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3783098174 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32f00a4ea0 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3783098174 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f32f8005560 con 0x561b54bc2a30 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f330fe29740 10 client.15867 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T21:03:03.669 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f330fe29740 10 client.15867 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f330fe29740 20 client.15867 populate_metadata read hostname 'smithi096' 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f330fe29740 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54c46060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f330fe29740 4 client.15867 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561b54b2b200 con 0x561b54bdda40 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f32fffff640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54c46060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f32ee7fc640 10 client.15867 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:03.670 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.664+0000 7f32fffff640 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54c46060 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42a6c6cc9c2eb19 server_cookie=be01f8116e8579a6 in_seq=0 out_seq=0 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f32ee7fc640 1 -- 172.21.15.96:0/3783098174 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561b54b2b200 con 0x561b54bdda40 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f32ee7fc640 10 client.15867 handle_command_reply: tid=0 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 client.15867 shutdown 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 msgr2=0x561b54bd0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bd0ee0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f32f8003a30 tx=0x7f32f8019970 comp rx=0 tx=0).stop 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 shutdown_connections 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x561b54bdda40 0x561b54c46060 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.821 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x561b54bcdc10 0x561b54c3bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561b54bc3350 0x561b54bd1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 --2- 172.21.15.96:0/3783098174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561b54bc2a30 0x561b54bd0ee0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 >> 172.21.15.96:0/3783098174 conn(0x561b54bbef40 msgr2=0x561b54bc5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f32ee7fc640 0 client.15867 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 shutdown_connections 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 1 -- 172.21.15.96:0/3783098174 wait complete. 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.820+0000 7f330fe29740 20 client.15867 trim_cache size 0 max 16384 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:{ 2024-04-25T21:03:03.822 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "message": "", 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout: "return_code": 0 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stdout:} 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f32f40a3dc0 msgr2=0x7f32f40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f32f40a3dc0 0x7f32f40a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f3300006010 tx=0x7f3300009800 comp rx=0 tx=0).stop 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 msgr2=0x7f3308199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f3308199a40 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f32f000d770 tx=0x7f32f000dc30 comp rx=0 tx=0).stop 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 shutdown_connections 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f32f40a3dc0 0x7f32f40a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f3308199fd0 0x7f33081a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.823 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3308106170 0x7f3308199a40 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 --2- 172.21.15.96:0/2318384945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f33081010f0 0x7f3308199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 >> 172.21.15.96:0/2318384945 conn(0x7f3308077fd0 msgr2=0x7f33080fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 shutdown_connections 2024-04-25T21:03:03.824 INFO:teuthology.task.background_exec.ubuntu@smithi096.front.sepia.ceph.com.smithi096.stderr:2024-04-25T21:03:03.821+0000 7f330d031640 1 -- 172.21.15.96:0/2318384945 wait complete. 2024-04-25T21:03:03.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:03 smithi096 ceph-mon[29288]: pgmap v166: 97 pgs: 97 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 547 op/s 2024-04-25T21:03:04.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:03 smithi097 ceph-mon[32667]: pgmap v166: 97 pgs: 97 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 547 op/s 2024-04-25T21:03:04.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:03 smithi139 ceph-mon[33027]: pgmap v166: 97 pgs: 97 active+clean; 66 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.9 MiB/s rd, 2.0 MiB/s wr, 547 op/s 2024-04-25T21:03:05.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:05 smithi096 ceph-mon[29288]: pgmap v167: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.9 MiB/s wr, 780 op/s 2024-04-25T21:03:06.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:05 smithi097 ceph-mon[32667]: pgmap v167: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.9 MiB/s wr, 780 op/s 2024-04-25T21:03:06.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:05 smithi139 ceph-mon[33027]: pgmap v167: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 2.9 MiB/s wr, 780 op/s 2024-04-25T21:03:06.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:06 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:07.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:06 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:07.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:06 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:07.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:07 smithi096 ceph-mon[29288]: pgmap v168: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 362 op/s 2024-04-25T21:03:08.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:07 smithi097 ceph-mon[32667]: pgmap v168: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 362 op/s 2024-04-25T21:03:08.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:07 smithi139 ceph-mon[33027]: pgmap v168: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 362 op/s 2024-04-25T21:03:09.665 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-04-25T21:03:09.665 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-25T21:03:09.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:09 smithi096 ceph-mon[29288]: pgmap v169: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 362 op/s 2024-04-25T21:03:10.022 DEBUG:teuthology.parallel:result is None 2024-04-25T21:03:10.022 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T21:03:10.058 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-25T21:03:10.058 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-25T21:03:10.070 INFO:tasks.quiescer:joining Quiescers 2024-04-25T21:03:10.070 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-25T21:03:10.071 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:03:10.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:09 smithi097 ceph-mon[32667]: pgmap v169: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 362 op/s 2024-04-25T21:03:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:09 smithi139 ceph-mon[33027]: pgmap v169: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.8 MiB/s wr, 362 op/s 2024-04-25T21:03:10.338 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:03:11.974 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:11 smithi096 ceph-mon[29288]: pgmap v170: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2024-04-25T21:03:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 -- 172.21.15.96:0/3838408999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90103450 msgr2=0x7fcd9010f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:12.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/3838408999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90103450 0x7fcd9010f8e0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fcd8800b0a0 tx=0x7fcd8800b3b0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 -- 172.21.15.96:0/3838408999 shutdown_connections 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/3838408999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90103450 0x7fcd9010f8e0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/3838408999 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90108a80 0x7fcd90108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/3838408999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90102a90 0x7fcd90102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 -- 172.21.15.96:0/3838408999 >> 172.21.15.96:0/3838408999 conn(0x7fcd90079bf0 msgr2=0x7fcd9007a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 -- 172.21.15.96:0/3838408999 shutdown_connections 2024-04-25T21:03:12.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 -- 172.21.15.96:0/3838408999 wait complete. 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.074+0000 7fcd95e2c700 1 Processor -- start 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 -- start start 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90102a90 0x7fcd9019b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90103450 0x7fcd9019b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 0x7fcd9019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112c90 con 0x7fcd90108a80 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112b10 con 0x7fcd90103450 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd95e2c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fcd90112e10 con 0x7fcd90102a90 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 0x7fcd9019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 0x7fcd9019fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55674/0 (socket says 172.21.15.96:55674) 2024-04-25T21:03:12.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 -- 172.21.15.96:0/2843635280 learned_addr learned my addr 172.21.15.96:0/2843635280 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8effd700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90103450 0x7fcd9019b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8f7fe700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90102a90 0x7fcd9019b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 -- 172.21.15.96:0/2843635280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90102a90 msgr2=0x7fcd9019b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90102a90 0x7fcd9019b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 -- 172.21.15.96:0/2843635280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90103450 msgr2=0x7fcd9019b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90103450 0x7fcd9019b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8ffff700 1 -- 172.21.15.96:0/2843635280 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd901a0370 con 0x7fcd90108a80 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.075+0000 7fcd8f7fe700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90102a90 0x7fcd9019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.076+0000 7fcd8ffff700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 0x7fcd9019fc30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcd880062a0 tx=0x7fcd88004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.076+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88021070 con 0x7fcd90108a80 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.076+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd88003c00 con 0x7fcd90108a80 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.076+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd901a0600 con 0x7fcd90108a80 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.076+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd88004880 con 0x7fcd90108a80 2024-04-25T21:03:12.078 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.076+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901a7ee0 con 0x7fcd90108a80 2024-04-25T21:03:12.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.077+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd901041b0 con 0x7fcd90108a80 2024-04-25T21:03:12.079 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.078+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcd88004020 con 0x7fcd90108a80 2024-04-25T21:03:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.079+0000 7fcd8cff9700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd780a3db0 0x7fcd780a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.079+0000 7fcd8f7fe700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd780a3db0 0x7fcd780a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.079+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7fcd880d1cb0 con 0x7fcd90108a80 2024-04-25T21:03:12.080 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.079+0000 7fcd8f7fe700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd780a3db0 0x7fcd780a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd800099d0 tx=0x7fcd80008040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:12.084 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.084+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcd8809e450 con 0x7fcd90108a80 2024-04-25T21:03:12.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:11 smithi097 ceph-mon[32667]: pgmap v170: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2024-04-25T21:03:12.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:11 smithi139 ceph-mon[33027]: pgmap v170: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2024-04-25T21:03:12.326 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.326+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd901a80b0 con 0x7fcd90108a80 2024-04-25T21:03:12.327 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.326+0000 7fcd8cff9700 1 -- 172.21.15.96:0/2843635280 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11868 (secure 0 0 0) 0x7fcd8809dd50 con 0x7fcd90108a80 2024-04-25T21:03:12.327 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:03:12.327 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T21:03:05.678299+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25093":{"gid":25093,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2108619178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2108619178},{"type":"v1","addr":"172.21.15.139:6837","nonce":2108619178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24443,24422]},"id":1}]} 2024-04-25T21:03:12.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd780a3db0 msgr2=0x7fcd780a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:12.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd780a3db0 0x7fcd780a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcd800099d0 tx=0x7fcd80008040 comp rx=0 tx=0).stop 2024-04-25T21:03:12.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 msgr2=0x7fcd9019fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:12.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 0x7fcd9019fc30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcd880062a0 tx=0x7fcd88004dc0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 shutdown_connections 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fcd780a3db0 0x7fcd780a6270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd90108a80 0x7fcd9019fc30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fcd90102a90 0x7fcd9019b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 --2- 172.21.15.96:0/2843635280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcd90103450 0x7fcd9019b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.328+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 >> 172.21.15.96:0/2843635280 conn(0x7fcd90079bf0 msgr2=0x7fcd9010dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.329+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 shutdown_connections 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:12.329+0000 7fcd95e2c700 1 -- 172.21.15.96:0/2843635280 wait complete. 2024-04-25T21:03:12.330 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 32 2024-04-25T21:03:12.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:12 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2843635280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:03:13.094 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24443 quiesce db --cancel --all' 2024-04-25T21:03:13.094 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24443 quiesce db --cancel --all 2024-04-25T21:03:13.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:12 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2843635280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:03:13.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:12 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2843635280' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:03:13.336 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:03:13.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:13 smithi096 ceph-mon[29288]: pgmap v171: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 388 op/s 2024-04-25T21:03:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:13 smithi097 ceph-mon[32667]: pgmap v171: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 388 op/s 2024-04-25T21:03:14.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:13 smithi139 ceph-mon[33027]: pgmap v171: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 388 op/s 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.060+0000 7efd0a4d0700 1 -- 172.21.15.96:0/3295722594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 msgr2=0x7efd04102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.060+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/3295722594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04102f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009990 tx=0x7efcf8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.060+0000 7efd0a4d0700 1 -- 172.21.15.96:0/3295722594 shutdown_connections 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.060+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/3295722594 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 0x7efd0410f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.060+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/3295722594 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 0x7efd04108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.060+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/3295722594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04102f40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.061+0000 7efd0a4d0700 1 -- 172.21.15.96:0/3295722594 >> 172.21.15.96:0/3295722594 conn(0x7efd04079bb0 msgr2=0x7efd04079fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.061+0000 7efd0a4d0700 1 -- 172.21.15.96:0/3295722594 shutdown_connections 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.061+0000 7efd0a4d0700 1 -- 172.21.15.96:0/3295722594 wait complete. 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.061+0000 7efd0a4d0700 1 Processor -- start 2024-04-25T21:03:15.061 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.061+0000 7efd0a4d0700 1 -- start start 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 0x7efd041997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 0x7efd0419db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd04112d00 con 0x7efd04102ac0 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd04112b80 con 0x7efd04108a70 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd04112e80 con 0x7efd04103480 2024-04-25T21:03:15.063 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55700/0 (socket says 172.21.15.96:55700) 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 -- 172.21.15.96:0/4281244961 learned_addr learned my addr 172.21.15.96:0/4281244961 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd08a6d700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 0x7efd0419db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd037fe700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 0x7efd041997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 -- 172.21.15.96:0/4281244961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 msgr2=0x7efd041997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 0x7efd041997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 -- 172.21.15.96:0/4281244961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 msgr2=0x7efd0419db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 0x7efd0419db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 -- 172.21.15.96:0/4281244961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd0419e280 con 0x7efd04102ac0 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd08a6d700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 0x7efd0419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd037fe700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 0x7efd041997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd03fff700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04199270 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b530 tx=0x7efcf400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:15.064 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd017fa700 1 -- 172.21.15.96:0/4281244961 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4015020 con 0x7efd04102ac0 2024-04-25T21:03:15.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd017fa700 1 -- 172.21.15.96:0/4281244961 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf4011ce0 con 0x7efd04102ac0 2024-04-25T21:03:15.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd017fa700 1 -- 172.21.15.96:0/4281244961 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf4010d20 con 0x7efd04102ac0 2024-04-25T21:03:15.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.062+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd0419e570 con 0x7efd04102ac0 2024-04-25T21:03:15.065 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.063+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd041a5df0 con 0x7efd04102ac0 2024-04-25T21:03:15.067 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.064+0000 7efd017fa700 1 -- 172.21.15.96:0/4281244961 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efcf4018020 con 0x7efd04102ac0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.065+0000 7efd017fa700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7efcec0a3db0 0x7efcec0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.065+0000 7efd037fe700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7efcec0a3db0 0x7efcec0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.065+0000 7efd017fa700 1 -- 172.21.15.96:0/4281244961 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7efcf40cff50 con 0x7efd04102ac0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd037fe700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7efcec0a3db0 0x7efcec0a6270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efcf8000c00 tx=0x7efcf8005fb0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 Processor -- start 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 -- start start 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8da9d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daaccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab8d9f0d00 con 0x55ab8daad1f0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ab8d9979a0 con 0x55ab8da9dee0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd14d59b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ab8d9eccc0 con 0x55ab8da9d5c0 2024-04-25T21:03:15.068 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd08a6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd08a6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaf5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55708/0 (socket says 172.21.15.96:55708) 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd08a6d700 1 -- 172.21.15.96:0/1902974227 learned_addr learned my addr 172.21.15.96:0/1902974227 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.066+0000 7efd037fe700 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daaccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd03fff700 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8da9d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd08a6d700 1 -- 172.21.15.96:0/1902974227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 msgr2=0x55ab8da9d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd08a6d700 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8da9d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd08a6d700 1 -- 172.21.15.96:0/1902974227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 msgr2=0x55ab8daaccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd08a6d700 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daaccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd08a6d700 1 -- 172.21.15.96:0/1902974227 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab8d95b7b0 con 0x55ab8daad1f0 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd03fff700 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8da9d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efd08a6d700 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaf5e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c450 tx=0x7efcfc00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=616bca76d1c4d5ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:15.069 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efcea7fc700 1 -- 172.21.15.96:0/1902974227 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc00cb00 con 0x55ab8daad1f0 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efcea7fc700 1 -- 172.21.15.96:0/1902974227 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcfc00cca0 con 0x55ab8daad1f0 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.067+0000 7efcea7fc700 1 -- 172.21.15.96:0/1902974227 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc015620 con 0x55ab8daad1f0 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 -- 172.21.15.96:0/1902974227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 msgr2=0x55ab8daaf5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaf5e0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7efcfc00c450 tx=0x7efcfc00c480 comp rx=0 tx=0).stop 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 -- 172.21.15.96:0/1902974227 shutdown_connections 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaf5e0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8da9d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 --2- 172.21.15.96:0/1902974227 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daaccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 -- 172.21.15.96:0/1902974227 >> 172.21.15.96:0/1902974227 conn(0x55ab8d9ca900 msgr2=0x55ab8da99a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 -- 172.21.15.96:0/1902974227 shutdown_connections 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 -- 172.21.15.96:0/1902974227 wait complete. 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 -- start start 2024-04-25T21:03:15.070 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.068+0000 7efd14d59b80 1 Processor -- start 2024-04-25T21:03:15.073 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9d5c0 0x55ab8daac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9dee0 0x55ab8daa9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaa170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab8daafc80 con 0x55ab8daad1f0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x55ab8d9aa620 con 0x55ab8da9d5c0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x55ab8daafe00 con 0x55ab8da9dee0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 10 client.0 _ll_register_callbacks cb 0x55ab8d989990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55716/0 (socket says 172.21.15.96:55716) 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 -- 172.21.15.96:0/784102294 learned_addr learned my addr 172.21.15.96:0/784102294 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd03fff700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9d5c0 0x55ab8daac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd037fe700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9dee0 0x55ab8daa9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.074 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 -- 172.21.15.96:0/784102294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9dee0 msgr2=0x55ab8daa9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9dee0 0x55ab8daa9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 -- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9d5c0 msgr2=0x55ab8daac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9d5c0 0x55ab8daac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 -- 172.21.15.96:0/784102294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab8d93e940 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd08a6d700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaa170 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efcfc000f80 tx=0x7efcfc017790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cc045e3d386783 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efce37fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efce37fe700 1 -- 172.21.15.96:0/784102294 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc00cb00 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efce37fe700 1 -- 172.21.15.96:0/784102294 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7efcfc012040 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcfc00cca0 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efce37fe700 1 -- 172.21.15.96:0/784102294 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcfc0094f0 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.069+0000 7efd14d59b80 10 client.15885 fetch_fsmap learned FSMap version 32 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab8daba460 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efce37fe700 1 -- 172.21.15.96:0/784102294 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcfc00cca0 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efce37fe700 1 -- 172.21.15.96:0/784102294 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7efcfc00ace0 con 0x55ab8daad1f0 2024-04-25T21:03:15.075 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 10 client.15885 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 10 client.15885 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 20 client.15885 populate_metadata read hostname 'smithi096' 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8dabca40 0x55ab8dabee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 4 client.15885 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab8d9fe4f0 con 0x55ab8dabca40 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.070+0000 7efd03fff700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8dabca40 0x55ab8dabee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.071+0000 7efce37fe700 10 client.15885 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.071+0000 7efd03fff700 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8dabca40 0x55ab8dabee30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b088b5f8fcc8243 server_cookie=994a5dab5d703edb in_seq=0 out_seq=0 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.071+0000 7efce37fe700 1 -- 172.21.15.96:0/784102294 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ab8d9fe4f0 con 0x55ab8dabca40 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.071+0000 7efce37fe700 10 client.15885 handle_command_reply: tid=0 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.071+0000 7efd14d59b80 1 client.15885 shutdown 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 msgr2=0x55ab8daaa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaa170 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7efcfc000f80 tx=0x7efcfc017790 comp rx=0 tx=0).stop 2024-04-25T21:03:15.076 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 shutdown_connections 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8daaa170 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9d5c0 0x55ab8daac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8dabca40 0x55ab8dabee30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 --2- 172.21.15.96:0/784102294 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9dee0 0x55ab8daa9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 >> 172.21.15.96:0/784102294 conn(0x55ab8d9ca900 msgr2=0x55ab8da9ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efce37fe700 0 client.15885 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 shutdown_connections 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 1 -- 172.21.15.96:0/784102294 wait complete. 2024-04-25T21:03:15.077 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.072+0000 7efd14d59b80 20 client.15885 trim_cache size 0 max 16384 2024-04-25T21:03:15.091 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 Processor -- start 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 -- start start 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8dab8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 0x55ab8dac4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8daad1f0 0x55ab8dac49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd041adaa0 con 0x55ab8da9dee0 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd041ab2d0 con 0x55ab8daad1f0 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd14d59b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd041a9160 con 0x55ab8da9d5c0 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd037fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 0x55ab8dac4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd037fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 0x55ab8dac4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55718/0 (socket says 172.21.15.96:55718) 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.091+0000 7efd037fe700 1 -- 172.21.15.96:0/338561184 learned_addr learned my addr 172.21.15.96:0/338561184 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd08a6d700 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8daad1f0 0x55ab8dac49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd03fff700 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8dab8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd037fe700 1 -- 172.21.15.96:0/338561184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 msgr2=0x55ab8dab8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.092 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd037fe700 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8dab8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd037fe700 1 -- 172.21.15.96:0/338561184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8daad1f0 msgr2=0x55ab8dac49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd037fe700 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8daad1f0 0x55ab8dac49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd037fe700 1 -- 172.21.15.96:0/338561184 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab8daa81a0 con 0x55ab8da9dee0 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd03fff700 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8dab8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efd037fe700 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 0x55ab8dac4490 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efcf8015000 tx=0x7efcf8028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6b5c0bdd263107 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efce9ffb700 1 -- 172.21.15.96:0/338561184 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf80055a0 con 0x55ab8da9dee0 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efce9ffb700 1 -- 172.21.15.96:0/338561184 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf800bc80 con 0x55ab8da9dee0 2024-04-25T21:03:15.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.092+0000 7efce9ffb700 1 -- 172.21.15.96:0/338561184 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcf800fcb0 con 0x55ab8da9dee0 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 -- 172.21.15.96:0/338561184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 msgr2=0x55ab8dac4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 0x55ab8dac4490 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efcf8015000 tx=0x7efcf8028040 comp rx=0 tx=0).stop 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 -- 172.21.15.96:0/338561184 shutdown_connections 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8daad1f0 0x55ab8dac49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8dab8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 --2- 172.21.15.96:0/338561184 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8da9dee0 0x55ab8dac4490 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 -- 172.21.15.96:0/338561184 >> 172.21.15.96:0/338561184 conn(0x55ab8d9ca900 msgr2=0x55ab8dabea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 -- 172.21.15.96:0/338561184 shutdown_connections 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 -- 172.21.15.96:0/338561184 wait complete. 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.093+0000 7efd14d59b80 1 -- start start 2024-04-25T21:03:15.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 Processor -- start 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8daae2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8dab3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd041a97b0 con 0x55ab8daad1f0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7efd041aac00 con 0x55ab8da9dee0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7efd041a9e60 con 0x55ab8da9d5c0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd14d59b80 10 client.0 _ll_register_callbacks cb 0x55ab8d989990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8dab3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8dab3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:55732/0 (socket says 172.21.15.96:55732) 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 -- 172.21.15.96:0/16839030 learned_addr learned my addr 172.21.15.96:0/16839030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd03fff700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8daae2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd037fe700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 -- 172.21.15.96:0/16839030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 msgr2=0x55ab8daae2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.098 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8daae2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 -- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 msgr2=0x55ab8daae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.094+0000 7efd08a6d700 1 -- 172.21.15.96:0/16839030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab8daaf0b0 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd08a6d700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8dab3500 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009440 tx=0x7efcfc0098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49c87a706a9c484d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efce37fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efce37fe700 1 -- 172.21.15.96:0/16839030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcfc0178b0 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efce37fe700 1 -- 172.21.15.96:0/16839030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8411+0+0 (secure 0 0 0) 0x7efcfc01b040 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efcf800bc80 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efce37fe700 1 -- 172.21.15.96:0/16839030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7efcfc039050 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 10 client.15891 fetch_fsmap learned FSMap version 32 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ab8db23290 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efce37fe700 1 -- 172.21.15.96:0/16839030 <== mon.0 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efcf800bc80 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efce37fe700 1 -- 172.21.15.96:0/16839030 <== mon.0 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7efcfc01fba0 con 0x55ab8daad1f0 2024-04-25T21:03:15.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 10 client.15891 fetch_fsmap finished waiting for FSMap version 32 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 10 client.15891 resolve_mds: validated gid 24443 aka daemon mds.b 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 20 client.15891 populate_metadata read hostname 'smithi096' 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8db239d0 0x55ab8db25df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 4 client.15891 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.095+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 --> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55ab8d9f8590 con 0x55ab8db239d0 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.096+0000 7efd03fff700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8db239d0 0x55ab8db25df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.096+0000 7efce37fe700 10 client.15891 ms_handle_connect on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.096+0000 7efd03fff700 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8db239d0 0x55ab8db25df0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f524d008b33e233 server_cookie=3ce5b7dda77e81f0 in_seq=0 out_seq=0 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.096+0000 7efce37fe700 1 -- 172.21.15.96:0/16839030 <== mds.0 v2:172.21.15.97:6838/3461334961 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x55ab8d9f8590 con 0x55ab8db239d0 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.096+0000 7efce37fe700 10 client.15891 handle_command_reply: tid=0 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.096+0000 7efd14d59b80 1 client.15891 shutdown 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 msgr2=0x55ab8dab3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8dab3500 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7efcfc009440 tx=0x7efcfc0098e0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.100 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 shutdown_connections 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab8daad1f0 0x55ab8dab3500 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:6838/3461334961,v1:172.21.15.97:6839/3461334961] conn(0x55ab8db239d0 0x55ab8db25df0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x55ab8da9d5c0 0x55ab8daae2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 --2- 172.21.15.96:0/16839030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ab8da9dee0 0x55ab8daae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 >> 172.21.15.96:0/16839030 conn(0x55ab8d9ca900 msgr2=0x55ab8daa4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efce37fe700 0 client.15891 ms_handle_reset on v2:172.21.15.97:6838/3461334961 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 shutdown_connections 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 1 -- 172.21.15.96:0/16839030 wait complete. 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd14d59b80 20 client.15891 trim_cache size 0 max 16384 2024-04-25T21:03:15.101 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:03:15.102 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 32, 2024-04-25T21:03:15.102 INFO:teuthology.orchestra.run.smithi096.stdout: "leader": 24443, 2024-04-25T21:03:15.102 INFO:teuthology.orchestra.run.smithi096.stdout: "set_version": 6, 2024-04-25T21:03:15.102 INFO:teuthology.orchestra.run.smithi096.stdout: "sets": {} 2024-04-25T21:03:15.102 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7efcec0a3db0 msgr2=0x7efcec0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7efcec0a3db0 0x7efcec0a6270 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efcf8000c00 tx=0x7efcf8005fb0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 msgr2=0x7efd04199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.097+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04199270 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b530 tx=0x7efcf400b8f0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 shutdown_connections 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7efcec0a3db0 0x7efcec0a6270 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efd04108a70 0x7efd0419db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd04102ac0 0x7efd04199270 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 --2- 172.21.15.96:0/4281244961 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7efd04103480 0x7efd041997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 >> 172.21.15.96:0/4281244961 conn(0x7efd04079bb0 msgr2=0x7efd041019b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 shutdown_connections 2024-04-25T21:03:15.103 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:03:15.098+0000 7efd0a4d0700 1 -- 172.21.15.96:0/4281244961 wait complete. 2024-04-25T21:03:15.804 ERROR:teuthology.run_tasks:Manager failed: quiescer Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 418, in task stop_all_quiescers(ctx.ceph[cluster_name].thrashers) File "/home/teuthworker/src/github.com_ceph_ceph_e221b808d285b9d02e54cc10c53c84188e8e41e3/qa/tasks/quiescer.py", line 377, in stop_all_quiescers raise RuntimeError(f"error during quiesce thrashing: {thrasher.exception}") RuntimeError: error during quiesce thrashing: Error quiescing set '2c77974d': 110 (ETIMEDOUT) 2024-04-25T21:03:15.805 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-25T21:03:15.823 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T21:03:15.832 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-25T21:03:15.849 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-25T21:03:15.849 INFO:teuthology.task.exec:Executing custom commands... 2024-04-25T21:03:15.850 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T21:03:15.850 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-25T21:03:15.900 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-25T21:03:15.961 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:15 smithi096 ceph-mon[29288]: pgmap v172: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 425 op/s 2024-04-25T21:03:16.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:15 smithi097 ceph-mon[32667]: pgmap v172: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 425 op/s 2024-04-25T21:03:16.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:15 smithi139 ceph-mon[33027]: pgmap v172: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 425 op/s 2024-04-25T21:03:17.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:17 smithi096 ceph-mon[29288]: pgmap v173: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1022 KiB/s wr, 193 op/s 2024-04-25T21:03:18.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:17 smithi097 ceph-mon[32667]: pgmap v173: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1022 KiB/s wr, 193 op/s 2024-04-25T21:03:18.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:17 smithi139 ceph-mon[33027]: pgmap v173: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1022 KiB/s wr, 193 op/s 2024-04-25T21:03:19.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:19 smithi096 ceph-mon[29288]: pgmap v174: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1022 KiB/s wr, 193 op/s 2024-04-25T21:03:20.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:19 smithi097 ceph-mon[32667]: pgmap v174: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1022 KiB/s wr, 193 op/s 2024-04-25T21:03:20.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:19 smithi139 ceph-mon[33027]: pgmap v174: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 992 KiB/s rd, 1022 KiB/s wr, 193 op/s 2024-04-25T21:03:21.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:21 smithi096 ceph-mon[29288]: pgmap v175: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.0 MiB/s wr, 198 op/s 2024-04-25T21:03:22.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:21 smithi097 ceph-mon[32667]: pgmap v175: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.0 MiB/s wr, 198 op/s 2024-04-25T21:03:22.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:21 smithi139 ceph-mon[33027]: pgmap v175: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1001 KiB/s rd, 1.0 MiB/s wr, 198 op/s 2024-04-25T21:03:23.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:23 smithi096 ceph-mon[29288]: pgmap v176: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 44 KiB/s wr, 42 op/s 2024-04-25T21:03:24.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:23 smithi097 ceph-mon[32667]: pgmap v176: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 44 KiB/s wr, 42 op/s 2024-04-25T21:03:24.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:23 smithi139 ceph-mon[33027]: pgmap v176: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 44 KiB/s wr, 42 op/s 2024-04-25T21:03:25.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:25 smithi096 ceph-mon[29288]: pgmap v177: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 48 KiB/s wr, 46 op/s 2024-04-25T21:03:26.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:25 smithi097 ceph-mon[32667]: pgmap v177: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 48 KiB/s wr, 46 op/s 2024-04-25T21:03:26.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:25 smithi139 ceph-mon[33027]: pgmap v177: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 48 KiB/s wr, 46 op/s 2024-04-25T21:03:27.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:27 smithi096 ceph-mon[29288]: pgmap v178: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-25T21:03:28.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:27 smithi097 ceph-mon[32667]: pgmap v178: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-25T21:03:28.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:27 smithi139 ceph-mon[33027]: pgmap v178: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-25T21:03:29.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:29 smithi096 ceph-mon[29288]: pgmap v179: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-25T21:03:30.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:29 smithi097 ceph-mon[32667]: pgmap v179: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-25T21:03:30.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:29 smithi139 ceph-mon[33027]: pgmap v179: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-04-25T21:03:31.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:31 smithi096 ceph-mon[29288]: pgmap v180: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T21:03:32.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:31 smithi097 ceph-mon[32667]: pgmap v180: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T21:03:32.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:31 smithi139 ceph-mon[33027]: pgmap v180: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 12 op/s 2024-04-25T21:03:33.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:33 smithi096 ceph-mon[29288]: pgmap v181: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:03:34.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:33 smithi097 ceph-mon[32667]: pgmap v181: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:03:34.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:33 smithi139 ceph-mon[33027]: pgmap v181: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.7 KiB/s wr, 7 op/s 2024-04-25T21:03:35.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:34 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:35.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:34 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:35.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:34 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:03:36.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:35 smithi097 ceph-mon[32667]: pgmap v182: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-25T21:03:36.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:35 smithi139 ceph-mon[33027]: pgmap v182: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-25T21:03:36.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:35 smithi096 ceph-mon[29288]: pgmap v182: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-25T21:03:38.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:37 smithi097 ceph-mon[32667]: pgmap v183: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T21:03:38.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:37 smithi139 ceph-mon[33027]: pgmap v183: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T21:03:38.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:37 smithi096 ceph-mon[29288]: pgmap v183: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T21:03:40.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:39 smithi097 ceph-mon[32667]: pgmap v184: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T21:03:40.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:39 smithi139 ceph-mon[33027]: pgmap v184: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T21:03:40.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:39 smithi096 ceph-mon[29288]: pgmap v184: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-04-25T21:03:42.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:41 smithi097 ceph-mon[32667]: pgmap v185: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-25T21:03:42.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:41 smithi139 ceph-mon[33027]: pgmap v185: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-25T21:03:42.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:41 smithi096 ceph-mon[29288]: pgmap v185: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-04-25T21:03:44.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:43 smithi097 ceph-mon[32667]: pgmap v186: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T21:03:44.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:43 smithi139 ceph-mon[33027]: pgmap v186: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T21:03:44.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:43 smithi096 ceph-mon[29288]: pgmap v186: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 5 op/s 2024-04-25T21:03:46.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:45 smithi097 ceph-mon[32667]: pgmap v187: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T21:03:46.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:45 smithi139 ceph-mon[33027]: pgmap v187: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T21:03:46.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:45 smithi096 ceph-mon[29288]: pgmap v187: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-04-25T21:03:48.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:47 smithi097 ceph-mon[32667]: pgmap v188: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:48.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:47 smithi139 ceph-mon[33027]: pgmap v188: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:48.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:47 smithi096 ceph-mon[29288]: pgmap v188: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:50.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:49 smithi097 ceph-mon[32667]: pgmap v189: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:50.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:50.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:50.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:50.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:49 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:03:50.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:49 smithi139 ceph-mon[33027]: pgmap v189: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:50.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:50.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:50.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:50.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:49 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:03:50.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:49 smithi096 ceph-mon[29288]: pgmap v189: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:50.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:03:50.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:03:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:03:50.250 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:49 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:03:52.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:51 smithi097 ceph-mon[32667]: pgmap v190: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:03:52.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:51 smithi139 ceph-mon[33027]: pgmap v190: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:03:52.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:51 smithi096 ceph-mon[29288]: pgmap v190: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:03:54.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:53 smithi097 ceph-mon[32667]: pgmap v191: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:54.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:53 smithi139 ceph-mon[33027]: pgmap v191: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:54.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:53 smithi096 ceph-mon[29288]: pgmap v191: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-04-25T21:03:56.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:55 smithi097 ceph-mon[32667]: pgmap v192: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:03:56.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:55 smithi139 ceph-mon[33027]: pgmap v192: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:03:56.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:55 smithi096 ceph-mon[29288]: pgmap v192: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-04-25T21:03:58.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:57 smithi097 ceph-mon[32667]: pgmap v193: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:03:58.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:57 smithi139 ceph-mon[33027]: pgmap v193: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:03:58.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:57 smithi096 ceph-mon[29288]: pgmap v193: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:00.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:03:59 smithi097 ceph-mon[32667]: pgmap v194: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:00.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:03:59 smithi139 ceph-mon[33027]: pgmap v194: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:00.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:03:59 smithi096 ceph-mon[29288]: pgmap v194: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:02.051 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:01 smithi096 ceph-mon[29288]: pgmap v195: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:01 smithi097 ceph-mon[32667]: pgmap v195: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:02.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:01 smithi139 ceph-mon[33027]: pgmap v195: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:04.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:03 smithi097 ceph-mon[32667]: pgmap v196: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:04.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:03 smithi139 ceph-mon[33027]: pgmap v196: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:04.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:03 smithi096 ceph-mon[29288]: pgmap v196: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:06.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:05 smithi097 ceph-mon[32667]: pgmap v197: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:04:06.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:05 smithi139 ceph-mon[33027]: pgmap v197: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:04:06.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:05 smithi096 ceph-mon[29288]: pgmap v197: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:04:08.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:07 smithi097 ceph-mon[32667]: pgmap v198: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:04:08.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:07 smithi139 ceph-mon[33027]: pgmap v198: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:04:08.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:07 smithi096 ceph-mon[29288]: pgmap v198: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:04:10.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:09 smithi097 ceph-mon[32667]: pgmap v199: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:04:10.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:09 smithi097 ceph-mon[32667]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:04:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:09 smithi139 ceph-mon[33027]: pgmap v199: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:04:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:09 smithi139 ceph-mon[33027]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:04:10.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:09 smithi096 ceph-mon[29288]: pgmap v199: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:04:10.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:09 smithi096 ceph-mon[29288]: osdmap e96: 12 total, 12 up, 12 in 2024-04-25T21:04:12.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:11 smithi097 ceph-mon[32667]: pgmap v201: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:04:12.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:11 smithi139 ceph-mon[33027]: pgmap v201: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:04:12.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:11 smithi096 ceph-mon[29288]: pgmap v201: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:04:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:13 smithi097 ceph-mon[32667]: pgmap v202: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:04:14.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:13 smithi139 ceph-mon[33027]: pgmap v202: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:04:14.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:13 smithi096 ceph-mon[29288]: pgmap v202: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:04:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:16 smithi096 ceph-mon[29288]: pgmap v203: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:04:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:16 smithi096 ceph-mon[29288]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:04:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:16 smithi097 ceph-mon[32667]: pgmap v203: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:04:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:16 smithi097 ceph-mon[32667]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:04:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:16 smithi139 ceph-mon[33027]: pgmap v203: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:04:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:16 smithi139 ceph-mon[33027]: osdmap e97: 12 total, 12 up, 12 in 2024-04-25T21:04:18.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:18 smithi096 ceph-mon[29288]: pgmap v205: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:04:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:18 smithi097 ceph-mon[32667]: pgmap v205: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:04:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:18 smithi139 ceph-mon[33027]: pgmap v205: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:04:20.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:20 smithi096 ceph-mon[29288]: pgmap v206: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-04-25T21:04:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:20 smithi097 ceph-mon[32667]: pgmap v206: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-04-25T21:04:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:20 smithi139 ceph-mon[33027]: pgmap v206: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 208 B/s wr, 6 op/s 2024-04-25T21:04:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:22 smithi096 ceph-mon[29288]: pgmap v207: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:04:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:22 smithi097 ceph-mon[32667]: pgmap v207: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:04:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:22 smithi139 ceph-mon[33027]: pgmap v207: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:04:24.415 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:24 smithi097 ceph-mon[32667]: pgmap v208: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:04:24.424 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:24 smithi139 ceph-mon[33027]: pgmap v208: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:04:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:24 smithi096 ceph-mon[29288]: pgmap v208: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:04:26.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:26 smithi096 ceph-mon[29288]: pgmap v209: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:04:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:26 smithi097 ceph-mon[32667]: pgmap v209: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:04:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:26 smithi139 ceph-mon[33027]: pgmap v209: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:04:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:28 smithi096 ceph-mon[29288]: pgmap v210: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T21:04:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:28 smithi097 ceph-mon[32667]: pgmap v210: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T21:04:28.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:28 smithi139 ceph-mon[33027]: pgmap v210: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T21:04:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:30 smithi096 ceph-mon[29288]: pgmap v211: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:30.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:30 smithi097 ceph-mon[32667]: pgmap v211: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:30 smithi139 ceph-mon[33027]: pgmap v211: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:32.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:32 smithi096 ceph-mon[29288]: pgmap v212: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:32 smithi097 ceph-mon[32667]: pgmap v212: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:32 smithi139 ceph-mon[33027]: pgmap v212: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:34 smithi096 ceph-mon[29288]: pgmap v213: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:34.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:34 smithi097 ceph-mon[32667]: pgmap v213: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:34 smithi139 ceph-mon[33027]: pgmap v213: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:36.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:36 smithi096 ceph-mon[29288]: pgmap v214: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:36 smithi097 ceph-mon[32667]: pgmap v214: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:36 smithi139 ceph-mon[33027]: pgmap v214: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:38 smithi096 ceph-mon[29288]: pgmap v215: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:38 smithi097 ceph-mon[32667]: pgmap v215: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:38 smithi139 ceph-mon[33027]: pgmap v215: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:40.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:40 smithi096 ceph-mon[29288]: pgmap v216: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:40.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:40 smithi097 ceph-mon[32667]: pgmap v216: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:40 smithi139 ceph-mon[33027]: pgmap v216: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:42.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:42 smithi096 ceph-mon[29288]: pgmap v217: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:42 smithi097 ceph-mon[32667]: pgmap v217: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:42 smithi139 ceph-mon[33027]: pgmap v217: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:44.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:44 smithi096 ceph-mon[29288]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:44.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:44 smithi097 ceph-mon[32667]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:44.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:44 smithi139 ceph-mon[33027]: pgmap v218: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:46.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:46 smithi096 ceph-mon[29288]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:46 smithi097 ceph-mon[32667]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:46 smithi139 ceph-mon[33027]: pgmap v219: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:48.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:48 smithi096 ceph-mon[29288]: pgmap v220: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:48.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:48 smithi097 ceph-mon[32667]: pgmap v220: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:48 smithi139 ceph-mon[33027]: pgmap v220: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:50.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:50 smithi096 ceph-mon[29288]: pgmap v221: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:50.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:50 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:50 smithi097 ceph-mon[32667]: pgmap v221: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:50.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:50 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:50 smithi139 ceph-mon[33027]: pgmap v221: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:50 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:04:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:51.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:04:51.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:51.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:51.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:04:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:04:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:04:51.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:04:52.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:52 smithi097 ceph-mon[32667]: pgmap v222: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:52 smithi139 ceph-mon[33027]: pgmap v222: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:52 smithi096 ceph-mon[29288]: pgmap v222: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:54.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:54 smithi097 ceph-mon[32667]: pgmap v223: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:54.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:54 smithi139 ceph-mon[33027]: pgmap v223: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:54.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:54 smithi096 ceph-mon[29288]: pgmap v223: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:56.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:56 smithi097 ceph-mon[32667]: pgmap v224: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:56.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:56 smithi139 ceph-mon[33027]: pgmap v224: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:56 smithi096 ceph-mon[29288]: pgmap v224: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:04:58.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:04:58 smithi097 ceph-mon[32667]: pgmap v225: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:58.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:04:58 smithi139 ceph-mon[33027]: pgmap v225: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:04:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:04:58 smithi096 ceph-mon[29288]: pgmap v225: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:00.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:00 smithi097 ceph-mon[32667]: pgmap v226: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:00 smithi139 ceph-mon[33027]: pgmap v226: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:00.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:00 smithi096 ceph-mon[29288]: pgmap v226: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:02.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:02 smithi097 ceph-mon[32667]: pgmap v227: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:02 smithi139 ceph-mon[33027]: pgmap v227: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:02 smithi096 ceph-mon[29288]: pgmap v227: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:04.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:04 smithi097 ceph-mon[32667]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:04 smithi139 ceph-mon[33027]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:04 smithi096 ceph-mon[29288]: pgmap v228: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:06.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:06 smithi097 ceph-mon[32667]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:06 smithi139 ceph-mon[33027]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:06 smithi096 ceph-mon[29288]: pgmap v229: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:08.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:08 smithi097 ceph-mon[32667]: pgmap v230: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:08.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:08 smithi139 ceph-mon[33027]: pgmap v230: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:08 smithi096 ceph-mon[29288]: pgmap v230: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:10 smithi097 ceph-mon[32667]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:10.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:10 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:10 smithi139 ceph-mon[33027]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:10.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:10 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:10 smithi096 ceph-mon[29288]: pgmap v231: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:10 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:05:12.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:12 smithi139 ceph-mon[33027]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:12 smithi096 ceph-mon[29288]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:12.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:12 smithi097 ceph-mon[32667]: pgmap v232: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:14.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:14 smithi139 ceph-mon[33027]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:14.750 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:14 smithi096 ceph-mon[29288]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:14 smithi097 ceph-mon[32667]: pgmap v233: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-04-25T21:05:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:16 smithi139 ceph-mon[33027]: pgmap v234: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:16 smithi096 ceph-mon[29288]: pgmap v234: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:16.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:16 smithi097 ceph-mon[32667]: pgmap v234: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-04-25T21:05:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:18 smithi139 ceph-mon[33027]: pgmap v235: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:05:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:18 smithi096 ceph-mon[29288]: pgmap v235: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:05:18.866 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:18 smithi097 ceph-mon[32667]: pgmap v235: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:05:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:20 smithi096 ceph-mon[29288]: pgmap v236: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:05:20.865 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:20 smithi097 ceph-mon[32667]: pgmap v236: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:05:20.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:20 smithi139 ceph-mon[33027]: pgmap v236: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-04-25T21:05:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:22 smithi096 ceph-mon[29288]: pgmap v237: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:05:22.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:22 smithi097 ceph-mon[32667]: pgmap v237: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:05:22.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:22 smithi139 ceph-mon[33027]: pgmap v237: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2024-04-25T21:05:24.665 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:24 smithi097 ceph-mon[32667]: pgmap v238: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:24.674 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:24 smithi139 ceph-mon[33027]: pgmap v238: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:24 smithi096 ceph-mon[29288]: pgmap v238: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:26 smithi096 ceph-mon[29288]: pgmap v239: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:26.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:26 smithi097 ceph-mon[32667]: pgmap v239: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:26.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:26 smithi139 ceph-mon[33027]: pgmap v239: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:28 smithi096 ceph-mon[29288]: pgmap v240: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:28 smithi097 ceph-mon[32667]: pgmap v240: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:28.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:28 smithi139 ceph-mon[33027]: pgmap v240: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:30.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:30 smithi096 ceph-mon[29288]: pgmap v241: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:30.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:30 smithi097 ceph-mon[32667]: pgmap v241: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:30 smithi139 ceph-mon[33027]: pgmap v241: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:32 smithi096 ceph-mon[29288]: pgmap v242: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:32.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:32 smithi097 ceph-mon[32667]: pgmap v242: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:32 smithi139 ceph-mon[33027]: pgmap v242: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:34.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:34 smithi096 ceph-mon[29288]: pgmap v243: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:34.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:34 smithi097 ceph-mon[32667]: pgmap v243: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:34.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:34 smithi139 ceph-mon[33027]: pgmap v243: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:36.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:36 smithi096 ceph-mon[29288]: pgmap v244: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:36.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:36 smithi097 ceph-mon[32667]: pgmap v244: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:36.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:36 smithi139 ceph-mon[33027]: pgmap v244: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:38.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:38 smithi096 ceph-mon[29288]: pgmap v245: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:38.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:38 smithi097 ceph-mon[32667]: pgmap v245: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:38.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:38 smithi139 ceph-mon[33027]: pgmap v245: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:40.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:40 smithi096 ceph-mon[29288]: pgmap v246: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:40.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:40 smithi097 ceph-mon[32667]: pgmap v246: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:40 smithi139 ceph-mon[33027]: pgmap v246: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:42.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:42 smithi096 ceph-mon[29288]: pgmap v247: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:42.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:42 smithi097 ceph-mon[32667]: pgmap v247: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:42 smithi139 ceph-mon[33027]: pgmap v247: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:44 smithi096 ceph-mon[29288]: pgmap v248: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:44.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:44 smithi097 ceph-mon[32667]: pgmap v248: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:44.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:44 smithi139 ceph-mon[33027]: pgmap v248: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:46.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:46 smithi096 ceph-mon[29288]: pgmap v249: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:46.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:46 smithi097 ceph-mon[32667]: pgmap v249: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:46 smithi139 ceph-mon[33027]: pgmap v249: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:48.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:48 smithi096 ceph-mon[29288]: pgmap v250: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:48 smithi097 ceph-mon[32667]: pgmap v250: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:48 smithi139 ceph-mon[33027]: pgmap v250: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:50.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:50 smithi096 ceph-mon[29288]: pgmap v251: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:50.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:50 smithi097 ceph-mon[32667]: pgmap v251: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:50.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:50 smithi139 ceph-mon[33027]: pgmap v251: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:51.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:05:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:05:51.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:05:51.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:05:51.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:05:51.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:05:52.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:52 smithi096 ceph-mon[29288]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:52.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:52 smithi097 ceph-mon[32667]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:52 smithi139 ceph-mon[33027]: pgmap v252: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:54.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:54 smithi096 ceph-mon[29288]: pgmap v253: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:54 smithi097 ceph-mon[32667]: pgmap v253: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:54 smithi139 ceph-mon[33027]: pgmap v253: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:56.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:56 smithi096 ceph-mon[29288]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:56 smithi097 ceph-mon[32667]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:56.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:56 smithi139 ceph-mon[33027]: pgmap v254: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:05:58.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:05:58 smithi096 ceph-mon[29288]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:58.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:05:58 smithi097 ceph-mon[32667]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:05:58.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:05:58 smithi139 ceph-mon[33027]: pgmap v255: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:00.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:00 smithi096 ceph-mon[29288]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:00.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:00 smithi097 ceph-mon[32667]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:00.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:00 smithi139 ceph-mon[33027]: pgmap v256: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:02.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:02 smithi096 ceph-mon[29288]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:02.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:02 smithi097 ceph-mon[32667]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:02.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:02 smithi139 ceph-mon[33027]: pgmap v257: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:04.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:04 smithi096 ceph-mon[29288]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:04.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:04 smithi097 ceph-mon[32667]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:04.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:04 smithi139 ceph-mon[33027]: pgmap v258: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:06.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:06 smithi096 ceph-mon[29288]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:06:06.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:06 smithi097 ceph-mon[32667]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:06:06.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:06 smithi139 ceph-mon[33027]: pgmap v259: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:06:08.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:08 smithi096 ceph-mon[29288]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:06:08.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:08 smithi097 ceph-mon[32667]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:06:08.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:08 smithi139 ceph-mon[33027]: pgmap v260: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:06:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:10 smithi096 ceph-mon[29288]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:06:10.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:10 smithi096 ceph-mon[29288]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:06:10.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:10 smithi097 ceph-mon[32667]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:06:10.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:10 smithi097 ceph-mon[32667]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:06:10.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:10 smithi139 ceph-mon[33027]: pgmap v261: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:06:10.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:10 smithi139 ceph-mon[33027]: osdmap e98: 12 total, 12 up, 12 in 2024-04-25T21:06:12.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:12 smithi096 ceph-mon[29288]: pgmap v263: 97 pgs: 33 active+clean+snaptrim_wait, 47 active+clean, 17 active+clean+snaptrim; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:06:12.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:12 smithi097 ceph-mon[32667]: pgmap v263: 97 pgs: 33 active+clean+snaptrim_wait, 47 active+clean, 17 active+clean+snaptrim; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:06:12.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:12 smithi139 ceph-mon[33027]: pgmap v263: 97 pgs: 33 active+clean+snaptrim_wait, 47 active+clean, 17 active+clean+snaptrim; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:06:14.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:14 smithi096 ceph-mon[29288]: pgmap v264: 97 pgs: 33 active+clean+snaptrim_wait, 47 active+clean, 17 active+clean+snaptrim; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:06:14.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:14 smithi097 ceph-mon[32667]: pgmap v264: 97 pgs: 33 active+clean+snaptrim_wait, 47 active+clean, 17 active+clean+snaptrim; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:06:14.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:14 smithi139 ceph-mon[33027]: pgmap v264: 97 pgs: 33 active+clean+snaptrim_wait, 47 active+clean, 17 active+clean+snaptrim; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:06:16.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:16 smithi096 ceph-mon[29288]: pgmap v265: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:16.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:16 smithi097 ceph-mon[32667]: pgmap v265: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:16.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:16 smithi139 ceph-mon[33027]: pgmap v265: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:18.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:18 smithi096 ceph-mon[29288]: pgmap v266: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:18.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:18 smithi097 ceph-mon[32667]: pgmap v266: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:18.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:18 smithi139 ceph-mon[33027]: pgmap v266: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:20 smithi096 ceph-mon[29288]: pgmap v267: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:20.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:20 smithi096 ceph-mon[29288]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:06:20.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:20 smithi097 ceph-mon[32667]: pgmap v267: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:20.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:20 smithi097 ceph-mon[32667]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:06:20.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:20 smithi139 ceph-mon[33027]: pgmap v267: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:06:20.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:20 smithi139 ceph-mon[33027]: osdmap e99: 12 total, 12 up, 12 in 2024-04-25T21:06:22.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:22 smithi096 ceph-mon[29288]: pgmap v269: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:06:22.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:22 smithi097 ceph-mon[32667]: pgmap v269: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:06:22.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:22 smithi139 ceph-mon[33027]: pgmap v269: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:06:24.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:24 smithi096 ceph-mon[29288]: pgmap v270: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:06:24.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:24 smithi097 ceph-mon[32667]: pgmap v270: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:06:24.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:24 smithi139 ceph-mon[33027]: pgmap v270: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:06:26.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:26 smithi096 ceph-mon[29288]: pgmap v271: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:26.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:26 smithi097 ceph-mon[32667]: pgmap v271: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:26.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:26 smithi139 ceph-mon[33027]: pgmap v271: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:27.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:27 smithi096 ceph-mon[29288]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:06:27.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:27 smithi097 ceph-mon[32667]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:06:27.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:27 smithi139 ceph-mon[33027]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-25T21:06:28.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:28 smithi096 ceph-mon[29288]: pgmap v272: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:28.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:28 smithi097 ceph-mon[32667]: pgmap v272: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:28.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:28 smithi139 ceph-mon[33027]: pgmap v272: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:30.748 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:30 smithi096 ceph-mon[29288]: pgmap v273: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:30.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:30 smithi097 ceph-mon[32667]: pgmap v273: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:30.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:30 smithi139 ceph-mon[33027]: pgmap v273: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:06:32.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:32 smithi096 ceph-mon[29288]: pgmap v274: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:06:32.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:32 smithi097 ceph-mon[32667]: pgmap v274: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:06:32.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:32 smithi139 ceph-mon[33027]: pgmap v274: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:06:34.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:34 smithi097 ceph-mon[32667]: pgmap v275: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:34.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:34 smithi139 ceph-mon[33027]: pgmap v275: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:34.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:34 smithi096 ceph-mon[29288]: pgmap v275: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:36.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:36 smithi097 ceph-mon[32667]: pgmap v276: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:36.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:36 smithi139 ceph-mon[33027]: pgmap v276: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:36.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:36 smithi096 ceph-mon[29288]: pgmap v276: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:38.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:38 smithi097 ceph-mon[32667]: pgmap v277: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:38.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:38 smithi139 ceph-mon[33027]: pgmap v277: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:38.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:38 smithi096 ceph-mon[29288]: pgmap v277: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:40.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:40 smithi097 ceph-mon[32667]: pgmap v278: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:40.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:40 smithi139 ceph-mon[33027]: pgmap v278: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:40.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:40 smithi096 ceph-mon[29288]: pgmap v278: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:42.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:42 smithi097 ceph-mon[32667]: pgmap v279: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:42.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:42 smithi139 ceph-mon[33027]: pgmap v279: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:42.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:42 smithi096 ceph-mon[29288]: pgmap v279: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:44.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:44 smithi097 ceph-mon[32667]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:44.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:44 smithi139 ceph-mon[33027]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:44.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:44 smithi096 ceph-mon[29288]: pgmap v280: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:46.862 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:46 smithi097 ceph-mon[32667]: pgmap v281: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:46.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:46 smithi139 ceph-mon[33027]: pgmap v281: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:46.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:46 smithi096 ceph-mon[29288]: pgmap v281: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:48.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:48 smithi097 ceph-mon[32667]: pgmap v282: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:48.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:48 smithi139 ceph-mon[33027]: pgmap v282: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:48.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:48 smithi096 ceph-mon[29288]: pgmap v282: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:50.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:50 smithi097 ceph-mon[32667]: pgmap v283: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:50.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:50 smithi139 ceph-mon[33027]: pgmap v283: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:50.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:50 smithi096 ceph-mon[29288]: pgmap v283: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:51.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:51 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:51.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:51 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:51.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:51 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:06:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:52 smithi097 ceph-mon[32667]: pgmap v284: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:52 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:06:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:52 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:06:52.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:52 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:06:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:52 smithi139 ceph-mon[33027]: pgmap v284: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:52 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:06:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:52 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:06:52.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:52 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:06:52.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:52 smithi096 ceph-mon[29288]: pgmap v284: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:52.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:52 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:06:52.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:52 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:06:52.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:52 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:06:53.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:53 smithi097 ceph-mon[32667]: pgmap v285: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:53 smithi139 ceph-mon[33027]: pgmap v285: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:53.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:53 smithi096 ceph-mon[29288]: pgmap v285: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:55.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:55 smithi096 ceph-mon[29288]: pgmap v286: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:56.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:55 smithi097 ceph-mon[32667]: pgmap v286: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:56.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:55 smithi139 ceph-mon[33027]: pgmap v286: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:06:57.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:57 smithi096 ceph-mon[29288]: pgmap v287: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:58.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:57 smithi097 ceph-mon[32667]: pgmap v287: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:58.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:57 smithi139 ceph-mon[33027]: pgmap v287: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:06:59.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:06:59 smithi096 ceph-mon[29288]: pgmap v288: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:00.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:06:59 smithi097 ceph-mon[32667]: pgmap v288: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:00.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:06:59 smithi139 ceph-mon[33027]: pgmap v288: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:01.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:01 smithi139 ceph-mon[33027]: pgmap v289: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:01.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:01 smithi096 ceph-mon[29288]: pgmap v289: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:02.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:01 smithi097 ceph-mon[32667]: pgmap v289: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:03.998 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:03 smithi096 ceph-mon[29288]: pgmap v290: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:04.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:03 smithi097 ceph-mon[32667]: pgmap v290: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:04.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:03 smithi139 ceph-mon[33027]: pgmap v290: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:05.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:05 smithi096 ceph-mon[29288]: pgmap v291: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:07:06.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:05 smithi097 ceph-mon[32667]: pgmap v291: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:07:06.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:05 smithi139 ceph-mon[33027]: pgmap v291: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 24 op/s 2024-04-25T21:07:07.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:07 smithi096 ceph-mon[29288]: pgmap v292: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:08.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:07 smithi097 ceph-mon[32667]: pgmap v292: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:08.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:07 smithi139 ceph-mon[33027]: pgmap v292: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:10.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:09 smithi097 ceph-mon[32667]: pgmap v293: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:10.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:09 smithi097 ceph-mon[32667]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:07:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:09 smithi139 ceph-mon[33027]: pgmap v293: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:10.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:09 smithi139 ceph-mon[33027]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:07:10.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:09 smithi096 ceph-mon[29288]: pgmap v293: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2024-04-25T21:07:10.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:09 smithi096 ceph-mon[29288]: osdmap e100: 12 total, 12 up, 12 in 2024-04-25T21:07:12.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:11 smithi097 ceph-mon[32667]: pgmap v295: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:12.121 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:11 smithi139 ceph-mon[33027]: pgmap v295: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:12.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:11 smithi096 ceph-mon[29288]: pgmap v295: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:13 smithi097 ceph-mon[32667]: pgmap v296: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:14.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:13 smithi139 ceph-mon[33027]: pgmap v296: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:14.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:13 smithi096 ceph-mon[29288]: pgmap v296: 97 pgs: 10 active+clean+snaptrim, 23 active+clean+snaptrim_wait, 64 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 54 KiB/s wr, 26 op/s 2024-04-25T21:07:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:16 smithi096 ceph-mon[29288]: pgmap v297: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:07:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:16 smithi096 ceph-mon[29288]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:07:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:16 smithi097 ceph-mon[32667]: pgmap v297: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:07:16.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:16 smithi097 ceph-mon[32667]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:07:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:16 smithi139 ceph-mon[33027]: pgmap v297: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:07:16.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:16 smithi139 ceph-mon[33027]: osdmap e101: 12 total, 12 up, 12 in 2024-04-25T21:07:18.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:18 smithi096 ceph-mon[29288]: pgmap v299: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:07:18.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:18 smithi097 ceph-mon[32667]: pgmap v299: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:07:18.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:18 smithi139 ceph-mon[33027]: pgmap v299: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:07:20.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:20 smithi096 ceph-mon[29288]: pgmap v300: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-25T21:07:20.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:20 smithi097 ceph-mon[32667]: pgmap v300: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-25T21:07:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:20 smithi139 ceph-mon[33027]: pgmap v300: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 206 B/s wr, 6 op/s 2024-04-25T21:07:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:22 smithi096 ceph-mon[29288]: pgmap v301: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:22 smithi097 ceph-mon[32667]: pgmap v301: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:22 smithi139 ceph-mon[33027]: pgmap v301: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:24 smithi096 ceph-mon[29288]: pgmap v302: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:24.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:24 smithi097 ceph-mon[32667]: pgmap v302: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:24 smithi139 ceph-mon[33027]: pgmap v302: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-04-25T21:07:26.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:26 smithi096 ceph-mon[29288]: pgmap v303: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:07:26.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:26 smithi097 ceph-mon[32667]: pgmap v303: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:07:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:26 smithi139 ceph-mon[33027]: pgmap v303: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-04-25T21:07:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:28 smithi096 ceph-mon[29288]: pgmap v304: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T21:07:28.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:28 smithi097 ceph-mon[32667]: pgmap v304: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T21:07:28.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:28 smithi139 ceph-mon[33027]: pgmap v304: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-04-25T21:07:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:30 smithi096 ceph-mon[29288]: pgmap v305: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:30.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:30 smithi097 ceph-mon[32667]: pgmap v305: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:30 smithi139 ceph-mon[33027]: pgmap v305: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:32.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:32 smithi096 ceph-mon[29288]: pgmap v306: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:32.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:32 smithi097 ceph-mon[32667]: pgmap v306: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:32.621 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:32 smithi139 ceph-mon[33027]: pgmap v306: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:34 smithi096 ceph-mon[29288]: pgmap v307: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:34.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:34 smithi097 ceph-mon[32667]: pgmap v307: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:34.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:34 smithi139 ceph-mon[33027]: pgmap v307: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:36.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:36 smithi096 ceph-mon[29288]: pgmap v308: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:36.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:36 smithi097 ceph-mon[32667]: pgmap v308: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:36.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:36 smithi139 ceph-mon[33027]: pgmap v308: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:38.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:38 smithi096 ceph-mon[29288]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:38.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:38 smithi097 ceph-mon[32667]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:38.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:38 smithi139 ceph-mon[33027]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:40.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:40 smithi096 ceph-mon[29288]: pgmap v310: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:40.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:40 smithi097 ceph-mon[32667]: pgmap v310: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:40.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:40 smithi139 ceph-mon[33027]: pgmap v310: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:41.968 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-25T21:07:42.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 -- 172.21.15.96:0/2659987420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c101a90 msgr2=0x7fdc2c105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2659987420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c101a90 0x7fdc2c105ad0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fdc20009900 tx=0x7fdc2002a410 comp rx=0 tx=0).stop 2024-04-25T21:07:42.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 -- 172.21.15.96:0/2659987420 shutdown_connections 2024-04-25T21:07:42.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2659987420 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 0x7fdc2c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2659987420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c101a90 0x7fdc2c105ad0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.121 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2659987420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c1014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 -- 172.21.15.96:0/2659987420 >> 172.21.15.96:0/2659987420 conn(0x7fdc2c0771d0 msgr2=0x7fdc2c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 -- 172.21.15.96:0/2659987420 shutdown_connections 2024-04-25T21:07:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.120+0000 7fdc3175d640 1 -- 172.21.15.96:0/2659987420 wait complete. 2024-04-25T21:07:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 Processor -- start 2024-04-25T21:07:42.122 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 -- start start 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 0x7fdc2c1030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc2c1035f0 0x7fdc2c1a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1110f0 con 0x7fdc2c1010e0 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c110f70 con 0x7fdc2c1035f0 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc3175d640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c111270 con 0x7fdc2c1061b0 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc2affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc2affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47086/0 (socket says 172.21.15.96:47086) 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc2affd640 1 -- 172.21.15.96:0/2603129493 learned_addr learned my addr 172.21.15.96:0/2603129493 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.121+0000 7fdc2b7fe640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc2c1035f0 0x7fdc2c1a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2affd640 1 -- 172.21.15.96:0/2603129493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 msgr2=0x7fdc2c1030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.125 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2a7fc640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 0x7fdc2c1030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2affd640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 0x7fdc2c1030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2affd640 1 -- 172.21.15.96:0/2603129493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc2c1035f0 msgr2=0x7fdc2c1a6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2affd640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc2c1035f0 0x7fdc2c1a6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2affd640 1 -- 172.21.15.96:0/2603129493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c1a6f10 con 0x7fdc2c1010e0 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2a7fc640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 0x7fdc2c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc2affd640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c102b70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00eab0 tx=0x7fdc1c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c00ce40 con 0x7fdc2c1010e0 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c1a7200 con 0x7fdc2c1010e0 2024-04-25T21:07:42.126 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc1c011920 con 0x7fdc2c1010e0 2024-04-25T21:07:42.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1c010a30 con 0x7fdc2c1010e0 2024-04-25T21:07:42.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.122+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1a7750 con 0x7fdc2c1010e0 2024-04-25T21:07:42.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.123+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2c04fce0 con 0x7fdc2c1010e0 2024-04-25T21:07:42.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.124+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc1c010bd0 con 0x7fdc2c1010e0 2024-04-25T21:07:42.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.125+0000 7fdc13fff640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdc140a3df0 0x7fdc140a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.127 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.125+0000 7fdc2a7fc640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdc140a3df0 0x7fdc140a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.125+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7fdc1c0d0db0 con 0x7fdc2c1010e0 2024-04-25T21:07:42.128 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.125+0000 7fdc2a7fc640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdc140a3df0 0x7fdc140a62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000c00 tx=0x7fdc20031400 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:42.130 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.130+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdc1c016030 con 0x7fdc2c1010e0 2024-04-25T21:07:42.329 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.329+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fdc2c101a90 con 0x7fdc140a3df0 2024-04-25T21:07:42.336 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.336+0000 7fdc13fff640 1 -- 172.21.15.96:0/2603129493 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fdc2c101a90 con 0x7fdc140a3df0 2024-04-25T21:07:42.337 INFO:teuthology.orchestra.run.smithi096.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-25T00:00:00", "created": "2024-04-25T20:57:39", "first": "2024-04-25T20:58:00", "last": "2024-04-25T21:07:00", "last_pruned": "2024-04-25T21:07:00", "created_count": 10, "pruned_count": 3, "active": true} 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdc140a3df0 msgr2=0x7fdc140a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdc140a3df0 0x7fdc140a62b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdc20000c00 tx=0x7fdc20031400 comp rx=0 tx=0).stop 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 msgr2=0x7fdc2c102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c102b70 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c00eab0 tx=0x7fdc1c00ef70 comp rx=0 tx=0).stop 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 shutdown_connections 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fdc140a3df0 0x7fdc140a62b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.339 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdc2c1035f0 0x7fdc2c1a6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fdc2c1061b0 0x7fdc2c1030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 --2- 172.21.15.96:0/2603129493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdc2c1010e0 0x7fdc2c102b70 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.338+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 >> 172.21.15.96:0/2603129493 conn(0x7fdc2c0771d0 msgr2=0x7fdc2c06a9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.339+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 shutdown_connections 2024-04-25T21:07:42.340 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.339+0000 7fdc3175d640 1 -- 172.21.15.96:0/2603129493 wait complete. 2024-04-25T21:07:42.349 DEBUG:teuthology.orchestra.run.smithi096:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.448+0000 7f1568d08640 1 -- 172.21.15.96:0/2893680634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 msgr2=0x7f1564101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.448+0000 7f1568d08640 1 --2- 172.21.15.96:0/2893680634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f1564101620 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1554009900 tx=0x7f155402a400 comp rx=0 tx=0).stop 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- 172.21.15.96:0/2893680634 shutdown_connections 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 --2- 172.21.15.96:0/2893680634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564101b60 0x7f156410dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 --2- 172.21.15.96:0/2893680634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f1564101620 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 --2- 172.21.15.96:0/2893680634 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564106fb0 0x7f1564107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- 172.21.15.96:0/2893680634 >> 172.21.15.96:0/2893680634 conn(0x7f1564076c50 msgr2=0x7f1564077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:42.452 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- 172.21.15.96:0/2893680634 shutdown_connections 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- 172.21.15.96:0/2893680634 wait complete. 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 Processor -- start 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- start start 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f15641975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101b60 0x7f1564197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564106fb0 0x7f156419be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f15641110e0 con 0x7f15641011a0 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f1564110f60 con 0x7f1564106fb0 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.449+0000 7f1568d08640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f1564111260 con 0x7f1564101b60 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f15641975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.453 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f15641975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47098/0 (socket says 172.21.15.96:47098) 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 -- 172.21.15.96:0/487574993 learned_addr learned my addr 172.21.15.96:0/487574993 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f1561d9b640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101b60 0x7f1564197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 -- 172.21.15.96:0/487574993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101b60 msgr2=0x7f1564197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101b60 0x7f1564197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 -- 172.21.15.96:0/487574993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564106fb0 msgr2=0x7f156419be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f1562d9d640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564106fb0 0x7f156419be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564106fb0 0x7f156419be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 -- 172.21.15.96:0/487574993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156419c580 con 0x7f15641011a0 2024-04-25T21:07:42.454 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f1562d9d640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564106fb0 0x7f156419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f156259c640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f15641975e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f155800c890 tx=0x7f155800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558004220 con 0x7f15641011a0 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156419c870 con 0x7f15641011a0 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15580043c0 con 0x7f15641011a0 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1558004a00 con 0x7f15641011a0 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.450+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15641a4100 con 0x7f15641011a0 2024-04-25T21:07:42.455 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.451+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1564102550 con 0x7f15641011a0 2024-04-25T21:07:42.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.452+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15580079b0 con 0x7f15641011a0 2024-04-25T21:07:42.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.453+0000 7f15537fe640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f154c0a3dc0 0x7f154c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.453+0000 7f1561d9b640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f154c0a3dc0 0x7f154c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.453+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7f15580d0510 con 0x7f15641011a0 2024-04-25T21:07:42.456 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.454+0000 7f1561d9b640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f154c0a3dc0 0x7f154c0a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1554031440 tx=0x7f1554005e80 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:42.457 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.458+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1558014030 con 0x7f15641011a0 2024-04-25T21:07:42.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:42 smithi096 ceph-mon[29288]: pgmap v311: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:42.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:42 smithi097 ceph-mon[32667]: pgmap v311: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:42.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:42 smithi139 ceph-mon[33027]: pgmap v311: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:07:42.659 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.658+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 --> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1564062390 con 0x7f154c0a3dc0 2024-04-25T21:07:42.666 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.666+0000 7f15537fe640 1 -- 172.21.15.96:0/487574993 <== mgr.15147 v2:172.21.15.96:6800/3924141367 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1564062390 con 0x7f154c0a3dc0 2024-04-25T21:07:42.666 INFO:teuthology.orchestra.run.smithi096.stdout:/ 1m 6m3h 2024-04-25T21:07:42.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f154c0a3dc0 msgr2=0x7f154c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f154c0a3dc0 0x7f154c0a6280 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1554031440 tx=0x7f1554005e80 comp rx=0 tx=0).stop 2024-04-25T21:07:42.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 msgr2=0x7f15641975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.668 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f15641975e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f155800c890 tx=0x7f155800cd50 comp rx=0 tx=0).stop 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 shutdown_connections 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f154c0a3dc0 0x7f154c0a6280 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1564106fb0 0x7f156419be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f1564101b60 0x7f1564197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 --2- 172.21.15.96:0/487574993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15641011a0 0x7f15641975e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 >> 172.21.15.96:0/487574993 conn(0x7f1564076c50 msgr2=0x7f15640fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 shutdown_connections 2024-04-25T21:07:42.669 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.668+0000 7f1568d08640 1 -- 172.21.15.96:0/487574993 wait complete. 2024-04-25T21:07:42.679 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-25T21:07:42.690 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 -- 172.21.15.96:0/2585107367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c106160 msgr2=0x7f4e4c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 --2- 172.21.15.96:0/2585107367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c106160 0x7f4e4c10dd30 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4e44009ca0 tx=0x7f4e4402a8c0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 -- 172.21.15.96:0/2585107367 shutdown_connections 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 --2- 172.21.15.96:0/2585107367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c106160 0x7f4e4c10dd30 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 --2- 172.21.15.96:0/2585107367 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e4c101a40 0x7f4e4c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 --2- 172.21.15.96:0/2585107367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 0x7f4e4c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 -- 172.21.15.96:0/2585107367 >> 172.21.15.96:0/2585107367 conn(0x7f4e4c0771d0 msgr2=0x7f4e4c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 -- 172.21.15.96:0/2585107367 shutdown_connections 2024-04-25T21:07:42.792 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 -- 172.21.15.96:0/2585107367 wait complete. 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 Processor -- start 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.789+0000 7f4e52519640 1 -- start start 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 0x7f4e4c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e4c106160 0x7f4e4c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 0x7f4e4c19d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c1121e0 con 0x7f4e4c19cee0 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c112060 con 0x7f4e4c101090 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f4e4c112360 con 0x7f4e4c106160 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 0x7f4e4c19d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 0x7f4e4c19d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47108/0 (socket says 172.21.15.96:47108) 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 -- 172.21.15.96:0/1324372875 learned_addr learned my addr 172.21.15.96:0/1324372875 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e4b7fe640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e4c106160 0x7f4e4c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.793 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e4bfff640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 0x7f4e4c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 -- 172.21.15.96:0/1324372875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e4c106160 msgr2=0x7f4e4c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e4c106160 0x7f4e4c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 -- 172.21.15.96:0/1324372875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 msgr2=0x7f4e4c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 0x7f4e4c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 -- 172.21.15.96:0/1324372875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e4c19d9d0 con 0x7f4e4c19cee0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e4bfff640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 0x7f4e4c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e50ab6640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 0x7f4e4c19d390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005cc0 tx=0x7f4e4402f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e44003cf0 con 0x7f4e4c19cee0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e440044f0 con 0x7f4e4c19cee0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e4c19dc60 con 0x7f4e4c19cee0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.790+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4402fd10 con 0x7f4e4c19cee0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.791+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e4c1a5480 con 0x7f4e4c19cee0 2024-04-25T21:07:42.794 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.792+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e4c0691c0 con 0x7f4e4c19cee0 2024-04-25T21:07:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.793+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4e44004040 con 0x7f4e4c19cee0 2024-04-25T21:07:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.793+0000 7f4e497fa640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4e340a3d20 0x7f4e340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.793+0000 7f4e4bfff640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4e340a3d20 0x7f4e340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.793+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7f4e440f20d0 con 0x7f4e4c19cee0 2024-04-25T21:07:42.795 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.794+0000 7f4e4bfff640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4e340a3d20 0x7f4e340a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0058c0 tx=0x7f4e3c005ec0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:42.797 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.797+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e440be960 con 0x7f4e4c19cee0 2024-04-25T21:07:42.975 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.975+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4e4c199970 con 0x7f4e4c19cee0 2024-04-25T21:07:42.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.976+0000 7f4e497fa640 1 -- 172.21.15.96:0/1324372875 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+22989 (secure 0 0 0) 0x7f4e4c199970 con 0x7f4e4c19cee0 2024-04-25T21:07:42.977 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:07:42.977 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":101,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T21:07:15.190881+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":14,"snap_epoch":100,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2385969372":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/1528469114":"2024-04-26T20:57:38.516139+0000","172.21.15.96:6800/23858312":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/995634760":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6801/23858312":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/2934738303":"2024-04-26T20:57:38.516139+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":14,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4e340a3d20 msgr2=0x7f4e340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4e340a3d20 0x7f4e340a61e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4e3c0058c0 tx=0x7f4e3c005ec0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 msgr2=0x7f4e4c19d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 0x7f4e4c19d390 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4e44005cc0 tx=0x7f4e4402f470 comp rx=0 tx=0).stop 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 shutdown_connections 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f4e340a3d20 0x7f4e340a61e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4e4c19cee0 0x7f4e4c19d390 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f4e4c106160 0x7f4e4c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 --2- 172.21.15.96:0/1324372875 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4e4c101090 0x7f4e4c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 >> 172.21.15.96:0/1324372875 conn(0x7f4e4c0771d0 msgr2=0x7f4e4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:42.979 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 shutdown_connections 2024-04-25T21:07:42.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:42.978+0000 7f4e52519640 1 -- 172.21.15.96:0/1324372875 wait complete. 2024-04-25T21:07:42.990 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T20:49:37.227783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T20:54:24.897398+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '91', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T20:54:28.260817+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '100', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 100, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T21:07:42.990 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T21:07:43.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.090+0000 7f51acdc6640 1 -- 172.21.15.96:0/2069836350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 msgr2=0x7f51a8110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.090+0000 7f51acdc6640 1 --2- 172.21.15.96:0/2069836350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a8110c20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f51a0009ca0 tx=0x7f51a002a950 comp rx=0 tx=0).stop 2024-04-25T21:07:43.093 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- 172.21.15.96:0/2069836350 shutdown_connections 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 --2- 172.21.15.96:0/2069836350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a8110c20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 --2- 172.21.15.96:0/2069836350 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074b40 0x7f51a810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 --2- 172.21.15.96:0/2069836350 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 0x7f51a8074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- 172.21.15.96:0/2069836350 >> 172.21.15.96:0/2069836350 conn(0x7f51a80793a0 msgr2=0x7f51a80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- 172.21.15.96:0/2069836350 shutdown_connections 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- 172.21.15.96:0/2069836350 wait complete. 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 Processor -- start 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- start start 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 0x7f51a81a0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074b40 0x7f51a81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51a81143c0 con 0x7f51a810e830 2024-04-25T21:07:43.094 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8114240 con 0x7f51a8074b40 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.091+0000 7f51acdc6640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f51a8114540 con 0x7f51a8074220 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a819a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47122/0 (socket says 172.21.15.96:47122) 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 -- 172.21.15.96:0/3958895370 learned_addr learned my addr 172.21.15.96:0/3958895370 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a659c640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 0x7f51a81a0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a5d9b640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074b40 0x7f51a81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 -- 172.21.15.96:0/3958895370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 msgr2=0x7f51a81a0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 0x7f51a81a0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 -- 172.21.15.96:0/3958895370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074b40 msgr2=0x7f51a81a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074b40 0x7f51a81a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 -- 172.21.15.96:0/3958895370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51a819af80 con 0x7f51a810e830 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a659c640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 0x7f51a81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51a6d9d640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a819a6c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f51a002ae70 tx=0x7f51a002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.095 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a0034040 con 0x7f51a810e830 2024-04-25T21:07:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51a00308f0 con 0x7f51a810e830 2024-04-25T21:07:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51a00047c0 con 0x7f51a810e830 2024-04-25T21:07:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51a819b210 con 0x7f51a810e830 2024-04-25T21:07:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.092+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51a81a73a0 con 0x7f51a810e830 2024-04-25T21:07:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.093+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51a8075960 con 0x7f51a810e830 2024-04-25T21:07:43.096 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.094+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51a0037020 con 0x7f51a810e830 2024-04-25T21:07:43.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.095+0000 7f51977fe640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f51900a3d20 0x7f51900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.095+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7f51a00f16f0 con 0x7f51a810e830 2024-04-25T21:07:43.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.095+0000 7f51a659c640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f51900a3d20 0x7f51900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.097 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.095+0000 7f51a659c640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f51900a3d20 0x7f51900a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51980058c0 tx=0x7f5198005850 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.099 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.099+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f51a00bdee0 con 0x7f51a810e830 2024-04-25T21:07:43.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:43 smithi096 ceph-mon[29288]: from='client.15897 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:43.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:43 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1324372875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:07:43.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.279+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51a802d5c0 con 0x7f51a810e830 2024-04-25T21:07:43.278 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.279+0000 7f51977fe640 1 -- 172.21.15.96:0/3958895370 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) ==== 93+0+10 (secure 0 0 0) 0x7f51a00bd7e0 con 0x7f51a810e830 2024-04-25T21:07:43.278 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 1 2024-04-25T21:07:43.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f51900a3d20 msgr2=0x7f51900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f51900a3d20 0x7f51900a61e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f51980058c0 tx=0x7f5198005850 comp rx=0 tx=0).stop 2024-04-25T21:07:43.280 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 msgr2=0x7f51a819a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a819a6c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f51a002ae70 tx=0x7f51a002f5d0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 shutdown_connections 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f51900a3d20 0x7f51900a61e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51a810e830 0x7f51a819a6c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51a8074b40 0x7f51a81a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 --2- 172.21.15.96:0/3958895370 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f51a8074220 0x7f51a81a0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 >> 172.21.15.96:0/3958895370 conn(0x7f51a80793a0 msgr2=0x7f51a80fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 shutdown_connections 2024-04-25T21:07:43.281 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.281+0000 7f51acdc6640 1 -- 172.21.15.96:0/3958895370 wait complete. 2024-04-25T21:07:43.291 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T21:07:43.394 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 -- 172.21.15.96:0/1557882139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a01031f0 msgr2=0x7f36a01035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1557882139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a01031f0 0x7f36a01035d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f3698009ca0 tx=0x7f369802a9e0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 -- 172.21.15.96:0/1557882139 shutdown_connections 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1557882139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a010c530 0x7f36a010e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1557882139 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a0103b10 0x7f36a010bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1557882139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a01031f0 0x7f36a01035d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 -- 172.21.15.96:0/1557882139 >> 172.21.15.96:0/1557882139 conn(0x7f36a00771d0 msgr2=0x7f36a00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 -- 172.21.15.96:0/1557882139 shutdown_connections 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.393+0000 7f36a4c31640 1 -- 172.21.15.96:0/1557882139 wait complete. 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 Processor -- start 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 -- start start 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a01031f0 0x7f36a019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 0x7f36a019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.395 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a010c530 0x7f36a01985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0112220 con 0x7f36a0103b10 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01120a0 con 0x7f36a010c530 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f36a4c31640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01123a0 con 0x7f36a01031f0 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 0x7f36a019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 0x7f36a019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47142/0 (socket says 172.21.15.96:47142) 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 -- 172.21.15.96:0/1628036030 learned_addr learned my addr 172.21.15.96:0/1628036030 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369ed9d640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a010c530 0x7f36a01985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369e59c640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a01031f0 0x7f36a019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 -- 172.21.15.96:0/1628036030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a01031f0 msgr2=0x7f36a019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a01031f0 0x7f36a019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 -- 172.21.15.96:0/1628036030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a010c530 msgr2=0x7f36a01985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a010c530 0x7f36a01985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 -- 172.21.15.96:0/1628036030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a0198d20 con 0x7f36a0103b10 2024-04-25T21:07:43.396 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369e59c640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a01031f0 0x7f36a019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:07:43.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.394+0000 7f369dd9b640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 0x7f36a019e930 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f369400b4a0 tx=0x7f369400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.395+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694014070 con 0x7f36a0103b10 2024-04-25T21:07:43.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.395+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3694010ac0 con 0x7f36a0103b10 2024-04-25T21:07:43.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.395+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a0199010 con 0x7f36a0103b10 2024-04-25T21:07:43.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.395+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3694017c30 con 0x7f36a0103b10 2024-04-25T21:07:43.397 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.395+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01a52b0 con 0x7f36a0103b10 2024-04-25T21:07:43.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.396+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36a006ad90 con 0x7f36a0103b10 2024-04-25T21:07:43.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.397+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3694017070 con 0x7f36a0103b10 2024-04-25T21:07:43.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.397+0000 7f368f7fe640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f36880a3d20 0x7f36880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.397+0000 7f369e59c640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f36880a3d20 0x7f36880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.397+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7f36940cfda0 con 0x7f36a0103b10 2024-04-25T21:07:43.398 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.398+0000 7f369e59c640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f36880a3d20 0x7f36880a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3698009a60 tx=0x7f3698009800 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.400 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.401+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f369409c590 con 0x7f36a0103b10 2024-04-25T21:07:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.576+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f36a0107710 con 0x7f36a0103b10 2024-04-25T21:07:43.576 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.577+0000 7f368f7fe640 1 -- 172.21.15.96:0/1628036030 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) ==== 104+0+11 (secure 0 0 0) 0x7f369409be90 con 0x7f36a0103b10 2024-04-25T21:07:43.576 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 32 2024-04-25T21:07:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f36880a3d20 msgr2=0x7f36880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f36880a3d20 0x7f36880a61e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3698009a60 tx=0x7f3698009800 comp rx=0 tx=0).stop 2024-04-25T21:07:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 msgr2=0x7f36a019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 0x7f36a019e930 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f369400b4a0 tx=0x7f369400b960 comp rx=0 tx=0).stop 2024-04-25T21:07:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 shutdown_connections 2024-04-25T21:07:43.578 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f36880a3d20 0x7f36880a61e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f36a010c530 0x7f36a01985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36a0103b10 0x7f36a019e930 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 --2- 172.21.15.96:0/1628036030 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f36a01031f0 0x7f36a019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 >> 172.21.15.96:0/1628036030 conn(0x7f36a00771d0 msgr2=0x7f36a00fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 shutdown_connections 2024-04-25T21:07:43.579 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.579+0000 7f36a4c31640 1 -- 172.21.15.96:0/1628036030 wait complete. 2024-04-25T21:07:43.589 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T21:07:43.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:43 smithi097 ceph-mon[32667]: from='client.15897 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:43.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:43 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1324372875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:07:43.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:43 smithi139 ceph-mon[33027]: from='client.15897 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:43.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:43 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1324372875' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:07:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 -- 172.21.15.96:0/758571361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb81014a0 msgr2=0x7fedb8105240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 --2- 172.21.15.96:0/758571361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb81014a0 0x7fedb8105240 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7feda0009900 tx=0x7feda002a410 comp rx=0 tx=0).stop 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 -- 172.21.15.96:0/758571361 shutdown_connections 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 --2- 172.21.15.96:0/758571361 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedb8105970 0x7fedb810d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 --2- 172.21.15.96:0/758571361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb81014a0 0x7fedb8105240 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 --2- 172.21.15.96:0/758571361 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8100af0 0x7fedb8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 -- 172.21.15.96:0/758571361 >> 172.21.15.96:0/758571361 conn(0x7fedb80760e0 msgr2=0x7fedb80764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 -- 172.21.15.96:0/758571361 shutdown_connections 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.691+0000 7fedbda12640 1 -- 172.21.15.96:0/758571361 wait complete. 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 Processor -- start 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 -- start start 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 0x7fedb819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8105970 0x7fedb819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedb81985d0 0x7fedb8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fedb8112230 con 0x7fedb8100af0 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fedb81120b0 con 0x7fedb8105970 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedbda12640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fedb81123b0 con 0x7fedb81985d0 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 0x7fedb819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 0x7fedb819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47162/0 (socket says 172.21.15.96:47162) 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 -- 172.21.15.96:0/2844709997 learned_addr learned my addr 172.21.15.96:0/2844709997 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedaffff640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8105970 0x7fedb819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 -- 172.21.15.96:0/2844709997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedb81985d0 msgr2=0x7fedb8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedb81985d0 0x7fedb8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 -- 172.21.15.96:0/2844709997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8105970 msgr2=0x7fedb819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8105970 0x7fedb819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 -- 172.21.15.96:0/2844709997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb81990f0 con 0x7fedb8100af0 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedaffff640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8105970 0x7fedb819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:07:43.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.692+0000 7fedb6ffd640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 0x7fedb819e420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feda800d770 tx=0x7feda800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.693+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8017070 con 0x7fedb8100af0 2024-04-25T21:07:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.693+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda8009c30 con 0x7fedb8100af0 2024-04-25T21:07:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.693+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8003cc0 con 0x7fedb8100af0 2024-04-25T21:07:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.693+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb81993e0 con 0x7fedb8100af0 2024-04-25T21:07:43.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.693+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb81a56b0 con 0x7fedb8100af0 2024-04-25T21:07:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.694+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb80691c0 con 0x7fedb8100af0 2024-04-25T21:07:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.695+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7feda8003e60 con 0x7fedb8100af0 2024-04-25T21:07:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.695+0000 7fedbca10640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7feda40a3d20 0x7feda40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.696+0000 7fedaffff640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7feda40a3d20 0x7feda40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.696+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7feda80d0210 con 0x7fedb8100af0 2024-04-25T21:07:43.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.696+0000 7fedaffff640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7feda40a3d20 0x7feda40a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedb8199710 tx=0x7feda0031400 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.699 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.700+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feda809c8f0 con 0x7fedb8100af0 2024-04-25T21:07:43.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.879+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fedb81014a0 con 0x7fedb8100af0 2024-04-25T21:07:43.879 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.880+0000 7fedbca10640 1 -- 172.21.15.96:0/2844709997 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) ==== 100+0+11 (secure 0 0 0) 0x7feda8009de0 con 0x7fedb8100af0 2024-04-25T21:07:43.879 INFO:teuthology.orchestra.run.smithi096.stdout:pg_num: 64 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7feda40a3d20 msgr2=0x7feda40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7feda40a3d20 0x7feda40a61e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fedb8199710 tx=0x7feda0031400 comp rx=0 tx=0).stop 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 msgr2=0x7fedb819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 0x7fedb819e420 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feda800d770 tx=0x7feda800dc30 comp rx=0 tx=0).stop 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 shutdown_connections 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7feda40a3d20 0x7feda40a61e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fedb81985d0 0x7fedb8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.881 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fedb8105970 0x7fedb819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 --2- 172.21.15.96:0/2844709997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fedb8100af0 0x7fedb819e420 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 >> 172.21.15.96:0/2844709997 conn(0x7fedb80760e0 msgr2=0x7fedb80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 shutdown_connections 2024-04-25T21:07:43.882 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.882+0000 7fedbda12640 1 -- 172.21.15.96:0/2844709997 wait complete. 2024-04-25T21:07:43.892 DEBUG:teuthology.orchestra.run.smithi096:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-25T21:07:43.991 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.990+0000 7f2ca443c640 1 -- 172.21.15.96:0/3784121797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c1031e0 msgr2=0x7f2c9c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.990+0000 7f2ca443c640 1 --2- 172.21.15.96:0/3784121797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c1031e0 0x7f2c9c1035c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f2c98009900 tx=0x7f2c9802a510 comp rx=0 tx=0).stop 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- 172.21.15.96:0/3784121797 shutdown_connections 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 --2- 172.21.15.96:0/3784121797 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c10c520 0x7f2c9c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 --2- 172.21.15.96:0/3784121797 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c103b00 0x7f2c9c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 --2- 172.21.15.96:0/3784121797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c1031e0 0x7f2c9c1035c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- 172.21.15.96:0/3784121797 >> 172.21.15.96:0/3784121797 conn(0x7f2c9c077f50 msgr2=0x7f2c9c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- 172.21.15.96:0/3784121797 shutdown_connections 2024-04-25T21:07:43.992 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- 172.21.15.96:0/3784121797 wait complete. 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 Processor -- start 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- start start 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c1031e0 0x7f2c9c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 0x7f2c9c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c10c520 0x7f2c9c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c1121e0 con 0x7f2c9c103b00 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c112060 con 0x7f2c9c10c520 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca443c640 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f2c9c112360 con 0x7f2c9c1031e0 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca19d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 0x7f2c9c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca19d7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 0x7f2c9c1a0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47174/0 (socket says 172.21.15.96:47174) 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.991+0000 7f2ca19d7640 1 -- 172.21.15.96:0/1617761607 learned_addr learned my addr 172.21.15.96:0/1617761607 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:07:43.993 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca29d9640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c10c520 0x7f2c9c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca21d8640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c1031e0 0x7f2c9c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca19d7640 1 -- 172.21.15.96:0/1617761607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c1031e0 msgr2=0x7f2c9c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca19d7640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c1031e0 0x7f2c9c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca19d7640 1 -- 172.21.15.96:0/1617761607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c10c520 msgr2=0x7f2c9c19a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca19d7640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c10c520 0x7f2c9c19a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca19d7640 1 -- 172.21.15.96:0/1617761607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c9c19b070 con 0x7f2c9c103b00 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca19d7640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 0x7f2c9c1a0c20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d770 tx=0x7f2c8c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c017070 con 0x7f2c9c103b00 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c9c19b360 con 0x7f2c9c103b00 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c8c0054d0 con 0x7f2c9c103b00 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c004c30 con 0x7f2c9c103b00 2024-04-25T21:07:43.994 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.992+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c9c1a7530 con 0x7f2c9c103b00 2024-04-25T21:07:43.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.993+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c9c0698e0 con 0x7f2c9c103b00 2024-04-25T21:07:43.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.994+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2c8c004e10 con 0x7f2c9c103b00 2024-04-25T21:07:43.995 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.995+0000 7f2c937fe640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2c880a3d20 0x7f2c880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:07:43.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.995+0000 7f2ca21d8640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2c880a3d20 0x7f2c880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:07:43.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.995+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8455+0+0 (secure 0 0 0) 0x7f2c8c0d01e0 con 0x7f2c9c103b00 2024-04-25T21:07:43.996 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:43.995+0000 7f2ca21d8640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2c880a3d20 0x7f2c880a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c98031440 tx=0x7f2c98005b50 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:07:44.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.000+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2c8c09c270 con 0x7f2c9c103b00 2024-04-25T21:07:44.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.224+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f2c9c100460 con 0x7f2c9c103b00 2024-04-25T21:07:44.231 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.232+0000 7f2c937fe640 1 -- 172.21.15.96:0/1617761607 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746060 (secure 0 0 0) 0x7f2c8c0a0570 con 0x7f2c9c103b00 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "epoch": 26, 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "active_gid": 15147, 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "active_name": "x", 2024-04-25T21:07:44.242 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addrs": { 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6800", 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3924141367 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v1", 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:6801", 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3924141367 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:44.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "active_addr": "172.21.15.96:6801/3924141367", 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "active_change": "2024-04-25T20:57:38.516340+0000", 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "active_mgr_features": 4540701547738038271, 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "available": true, 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "standbys": [ 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 15141, 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "y", 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.244 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.245 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.246 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T21:07:44.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T21:07:44.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T21:07:44.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T21:07:44.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T21:07:44.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T21:07:44.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T21:07:44.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.260 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:07:44.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T21:07:44.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.267 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.268 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.271 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.273 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T21:07:44.274 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.279 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T21:07:44.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:07:44.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:07:44.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T21:07:44.287 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T21:07:44.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T21:07:44.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T21:07:44.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:07:44.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.299 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:07:44.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:07:44.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:07:44.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:07:44.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:07:44.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:07:44.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.309 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.311 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T21:07:44.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T21:07:44.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T21:07:44.315 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T21:07:44.316 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.317 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.319 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.320 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.322 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.326 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:07:44.329 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.330 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.332 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.333 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T21:07:44.334 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T21:07:44.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T21:07:44.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.337 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.342 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:07:44.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:07:44.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.351 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.353 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.356 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:07:44.360 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:07:44.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.362 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.368 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.370 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.373 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.374 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.381 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.386 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:07:44.389 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:07:44.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:07:44.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:07:44.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:07:44.393 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.394 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.396 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.397 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:07:44.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.401 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.406 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.408 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.409 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.412 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.415 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.416 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.419 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.420 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.423 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.425 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.426 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.428 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:07:44.434 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.438 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.439 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.440 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.448 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T21:07:44.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.450 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.451 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.452 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.456 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.458 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.460 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T21:07:44.461 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T21:07:44.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.463 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.464 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.465 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.469 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.470 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.472 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.473 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.474 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.475 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T21:07:44.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.480 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.481 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.482 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.483 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.484 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.486 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.487 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T21:07:44.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.490 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.494 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.502 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T21:07:44.504 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.505 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.506 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.508 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.510 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.511 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.512 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.514 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.515 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.517 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.518 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.520 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.522 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.523 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.525 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.527 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.531 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.535 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.540 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.543 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.544 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.545 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.548 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.549 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.551 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.552 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.553 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.557 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T21:07:44.558 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.560 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T21:07:44.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.563 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.566 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T21:07:44.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.572 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.573 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.575 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.577 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.578 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.580 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.581 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.584 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.587 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.588 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.589 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.590 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.591 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T21:07:44.592 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.593 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.594 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.597 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.598 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.599 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.600 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.601 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.602 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.603 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T21:07:44.604 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T21:07:44.605 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T21:07:44.606 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T21:07:44.607 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T21:07:44.608 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T21:07:44.609 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T21:07:44.610 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.611 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.612 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.613 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.614 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.615 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.616 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.617 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.618 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.619 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.620 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.621 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.622 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.623 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.624 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.625 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.626 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.627 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.628 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.629 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.630 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.631 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.632 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.633 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.634 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.635 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.636 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.637 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T21:07:44.638 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:07:44.639 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:07:44.640 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.641 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.642 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.643 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.644 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.645 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.646 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.647 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T21:07:44.648 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T21:07:44.649 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.650 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.651 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.652 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.654 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.655 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.656 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.657 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T21:07:44.658 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T21:07:44.659 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.660 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.661 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.663 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.664 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.665 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.666 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.667 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.668 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.669 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.670 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.671 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.672 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.673 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.674 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.675 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.676 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.677 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.678 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.679 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.680 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.681 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.682 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.683 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.684 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.685 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.686 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.687 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "gid": 24709, 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "z", 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "mgr_features": 4540701547738038271, 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T21:07:44.688 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.689 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.690 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.691 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.692 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.693 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.694 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.695 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.696 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.697 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.698 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.699 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.700 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.701 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.702 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.703 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.704 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.705 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.706 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.707 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:07:44.708 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:07:44.709 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.710 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.711 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.712 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.713 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.714 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.715 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.716 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.717 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.718 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T21:07:44.719 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.720 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T21:07:44.721 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:07:44.722 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.723 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:07:44.724 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.725 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.726 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.727 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.728 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.729 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.730 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T21:07:44.731 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T21:07:44.732 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T21:07:44.733 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T21:07:44.734 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T21:07:44.735 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T21:07:44.736 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T21:07:44.737 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.738 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.739 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.740 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.741 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.742 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T21:07:44.743 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T21:07:44.744 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.745 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.746 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T21:07:44.747 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T21:07:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T21:07:44.749 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.750 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.751 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.753 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.754 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.755 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.757 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.758 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.759 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.760 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.761 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.762 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.763 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.764 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.765 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.766 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.767 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.769 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.770 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:07:44.771 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.772 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.773 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.774 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.775 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.776 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.777 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.778 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.779 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.780 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.781 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.782 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:07:44.783 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.784 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.785 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.786 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.787 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.788 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.789 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.790 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.791 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.792 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.793 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.794 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.795 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.796 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.797 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.798 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.799 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.800 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.801 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.802 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.803 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.804 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.805 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.806 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.807 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:07:44.808 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:07:44.809 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:07:44.810 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:07:44.811 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:07:44.812 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.813 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:07:44.814 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.815 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.816 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.817 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.818 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.820 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.821 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:07:44.822 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:07:44.823 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:07:44.824 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:07:44.825 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:07:44.826 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:07:44.827 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:07:44.828 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:07:44.829 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:07:44.830 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.831 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:07:44.832 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:07:44.833 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.834 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:07:44.835 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:07:44.836 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:07:44.837 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:07:44.838 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:07:44.839 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:07:44.840 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:07:44.841 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.842 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:07:44.843 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.844 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.845 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.846 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.847 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.848 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.849 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.850 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.851 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.852 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.853 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.854 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.855 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.856 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.857 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.858 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T21:07:44.859 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.860 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.861 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.862 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.863 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.864 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.865 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.866 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.867 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.868 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.869 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.870 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.871 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T21:07:44.872 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.873 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.875 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.876 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.877 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.878 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.879 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.880 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.881 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.882 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.883 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.884 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.885 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T21:07:44.886 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.887 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.888 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.889 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T21:07:44.890 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.891 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T21:07:44.892 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.893 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.894 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.895 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.896 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.897 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.898 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.899 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.900 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.901 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.902 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.903 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T21:07:44.904 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.905 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.906 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.907 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.908 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.909 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.910 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.911 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.912 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.913 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.914 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T21:07:44.915 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.916 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.916 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T21:07:44.916 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:07:44.916 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.916 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.917 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.918 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.919 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.920 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.921 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.922 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.923 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.924 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.925 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.926 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.927 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.928 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.929 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T21:07:44.930 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.931 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.933 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.934 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.935 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.936 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.937 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.938 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.939 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.940 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.941 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.942 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.943 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.944 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.945 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.946 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.948 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.949 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.950 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.951 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.952 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.953 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.954 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.955 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.956 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.958 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.959 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T21:07:44.960 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T21:07:44.961 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.962 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.963 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.964 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.965 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.966 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.967 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.968 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.969 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.970 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.971 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.972 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.973 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.974 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.975 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.976 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.977 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.978 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.979 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:07:44.980 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.981 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.982 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.983 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.984 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.985 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.986 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.988 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.989 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.990 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T21:07:44.991 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T21:07:44.992 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.993 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.994 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.995 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.996 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.997 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:44.998 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:44.999 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.000 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.001 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.002 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:45.003 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.005 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T21:07:45.006 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.007 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.008 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.009 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.010 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.011 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.012 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.013 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.014 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.015 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.016 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.017 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T21:07:45.018 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.019 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.020 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.021 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.022 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:45.024 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:45.025 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.026 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.027 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.028 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.029 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.030 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.031 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.032 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.033 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.034 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.035 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.036 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.037 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.038 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.039 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.040 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.041 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.042 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.043 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.044 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.045 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T21:07:45.046 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.047 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.048 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.049 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.050 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.051 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.052 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.053 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.054 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T21:07:45.055 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T21:07:45.056 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.057 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.058 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.059 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.060 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.061 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.062 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.063 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.064 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.065 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.066 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.067 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.068 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.070 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.071 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.072 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.073 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.074 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.075 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.076 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T21:07:45.077 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.078 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.079 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.080 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.081 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.082 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.083 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.084 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.085 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.086 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.087 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:07:45.088 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:07:45.089 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:07:45.090 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.091 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.092 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.093 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.094 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.095 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.096 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.097 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.098 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.099 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.100 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.101 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.102 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.103 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.104 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.105 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.107 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.108 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.109 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.110 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.111 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.112 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.113 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.114 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.115 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.116 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.117 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.118 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.119 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.120 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.121 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.122 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.123 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.124 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.125 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.126 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.127 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.128 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.129 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.130 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.131 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.132 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.133 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.134 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "modules": [ 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard", 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "iostat", 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "nfs", 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "restful", 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "snap_schedule" 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "available_modules": [ 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.135 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "alerts", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.136 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How frequently to reexamine health status", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.137 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.138 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.140 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.141 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_destination": { 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_destination", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.142 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email address to send alerts to", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_from_name": { 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_from_name", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.143 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "Ceph", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Email From: name", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_host": { 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_host", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.144 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP server", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_password": { 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_password", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.145 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Password to authenticate with", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_port": { 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_port", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.146 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "465", 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP port", 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.147 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_sender": { 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_sender", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SMTP envelope sender", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_ssl": { 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_ssl", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.149 INFO:teuthology.orchestra.run.smithi096.stdout: "smtp_user": { 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "smtp_user", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "User to authenticate as", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.150 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "balancer", 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "active": { 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "active", 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.151 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically balance PGs across cluster", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_time": { 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_time", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.152 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0000", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "beginning time of day to automatically balance", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "begin_weekday": { 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "begin_weekday", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.153 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_max_iterations": { 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_max_iterations", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.154 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "25", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "250", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_metrics": { 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_metrics", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.155 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "pgs,objects,bytes", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "crush_compat_step": { 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crush_compat_step", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.156 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.5", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0.001", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "0.999", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "aggressiveness of optimization", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "end_time": { 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_time", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.157 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2359", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ending time of day to automatically balance", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "end_weekday": { 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "end_weekday", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.158 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "0", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "6", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.159 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.160 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.161 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.162 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.163 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "min_score": { 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_score", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "upmap", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "crush-compat", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "none", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "read", 2024-04-25T21:07:45.166 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap-read" 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Balancer mode", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_ids": { 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_ids", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.167 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.168 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_deviation": { 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_deviation", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:45.169 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "upmap_max_optimizations": { 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "upmap_max_optimizations", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.170 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm", 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.171 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_down_multiplier": { 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_down_multiplier", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-25T21:07:45.172 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_refresh_rate": { 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_refresh_rate", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "20", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.173 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "agent_starting_port": { 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "agent_starting_port", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4721", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.174 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_ptrace": { 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_ptrace", 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.175 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_interval": { 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_interval", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:45.176 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "autotune_memory_target_ratio": { 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "autotune_memory_target_ratio", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0.7", 2024-04-25T21:07:45.177 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm_log_destination": { 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cephadm_log_destination", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.178 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "file", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "file,syslog", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "syslog" 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "cgroups_split": { 2024-04-25T21:07:45.179 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cgroups_split", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.180 INFO:teuthology.orchestra.run.smithi096.stdout: "config_checks_enabled": { 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_checks_enabled", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "config_dashboard": { 2024-04-25T21:07:45.181 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_dashboard", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.182 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_alertmanager": { 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_alertmanager", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.183 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_base": { 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_base", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Container image name, without the tag", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.184 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_elasticsearch": { 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_elasticsearch", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "elasticsearch container image", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.185 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_grafana": { 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_grafana", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.186 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_haproxy": { 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_haproxy", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "HAproxy container image", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.187 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_agent": { 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_agent", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger agent container image", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.188 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_collector": { 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_collector", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger collector container image", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.189 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_jaeger_query": { 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_jaeger_query", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Jaeger query container image", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.190 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_keepalived": { 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_keepalived", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Keepalived container image", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_loki": { 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_loki", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Loki container image", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.192 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_node_exporter": { 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_node_exporter", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.193 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_nvmeof": { 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_nvmeof", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Nvme-of container image", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.194 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_prometheus": { 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_prometheus", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Prometheus container image", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.195 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_promtail": { 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_promtail", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Promtail container image", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.196 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_samba": { 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_samba", 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.197 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Samba/SMB container image", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "container_image_snmp_gateway": { 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_image_snmp_gateway", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.198 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "SNMP Gateway container image", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "container_init": { 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "container_init", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.199 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Run podman/docker with `--init`", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "daemon_cache_timeout": { 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "daemon_cache_timeout", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.200 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_cephadm_command_timeout": { 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_cephadm_command_timeout", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "900", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.201 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "default_registry": { 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "default_registry", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "docker.io", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.202 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "device_cache_timeout": { 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_cache_timeout", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1800", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.203 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache device inventory", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "device_enhanced_scan": { 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_enhanced_scan", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-25T21:07:45.204 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "facts_cache_timeout": { 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "facts_cache_timeout", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "seconds to cache host facts data", 2024-04-25T21:07:45.205 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "grafana_dashboards_path": { 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "grafana_dashboards_path", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-25T21:07:45.206 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "host_check_interval": { 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "host_check_interval", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to perform a host check", 2024-04-25T21:07:45.207 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "hw_monitoring": { 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hw_monitoring", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.208 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "inventory_list_all": { 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "inventory_list_all", 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.209 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.210 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "log_refresh_metadata": { 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_refresh_metadata", 2024-04-25T21:07:45.211 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.212 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.213 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.214 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.215 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.216 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "*", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.217 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "max_count_per_host": { 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_count_per_host", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.218 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of daemons per service per host", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "max_osd_draining_count": { 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_osd_draining_count", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.219 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "migration_current": { 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "migration_current", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.220 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "internal - do not modify", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "mode": { 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mode", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "root", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.221 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm-package", 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "root" 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "mode for remote execution of cephadm", 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "oob_default_addr": { 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "oob_default_addr", 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.222 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "169.254.1.1", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "prometheus_alerts_path": { 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus_alerts_path", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.223 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_insecure": { 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_insecure", 2024-04-25T21:07:45.224 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.225 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_password": { 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_password", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.226 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_url": { 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_url", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.227 INFO:teuthology.orchestra.run.smithi096.stdout: "registry_username": { 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "registry_username", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.228 INFO:teuthology.orchestra.run.smithi096.stdout: "secure_monitoring_stack": { 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "secure_monitoring_stack", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.229 INFO:teuthology.orchestra.run.smithi096.stdout: "service_discovery_port": { 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "service_discovery_port", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8765", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "cephadm service discovery port", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.230 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "ssh_config_file": { 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssh_config_file", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.231 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "use_agent": { 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_agent", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.232 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "use_repo_digest": { 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "use_repo_digest", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.233 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_failed_host_check": { 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_failed_host_check", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if the host check fails", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.234 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_daemons": { 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_daemons", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.235 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_on_stray_hosts": { 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_on_stray_hosts", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.236 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crash", 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.237 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.238 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.239 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.240 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.241 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "retain_interval": { 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retain_interval", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.242 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "31536000", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_recent_interval": { 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_recent_interval", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.243 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1209600", 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.244 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dashboard", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.245 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_HOST": { 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.246 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.247 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_ENABLED": { 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_ENABLED", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.248 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.249 INFO:teuthology.orchestra.run.smithi096.stdout: "ENABLE_BROWSABLE_API": { 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.250 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.251 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.252 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.253 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.254 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.255 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.256 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.257 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.258 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_PASSWORD": { 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.259 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.260 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.261 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_URL": { 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_URL", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.262 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_API_USERNAME": { 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.263 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.264 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.265 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.266 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.267 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "[]", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.268 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "MULTICLUSTER_CONFIG": { 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "{}", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.269 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_HOST": { 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.270 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.271 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.272 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.273 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.274 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.275 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.276 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.277 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.278 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_ENABLED": { 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.279 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.280 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.281 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.282 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "45", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.283 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ACCESS_KEY": { 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.284 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "admin", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.285 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SECRET_KEY": { 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.286 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "RGW_API_SSL_VERIFY": { 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.287 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "UNSAFE_TLS_v1_2": { 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.288 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.289 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.290 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.291 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "cross_origin_url": { 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cross_origin_url", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.292 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "crt_file": { 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "crt_file", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.293 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "debug": { 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "debug", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Enable/disable debug options", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.294 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "jwt_token_ttl": { 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "jwt_token_ttl", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "28800", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.295 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.296 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.297 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.298 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.299 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.300 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.301 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "motd": { 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "motd", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "The message of the day", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.302 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect_resolve_ip_addr": { 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "redirect_resolve_ip_addr", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.303 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.304 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8080", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.305 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.306 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl_server_port": { 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl_server_port", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8443", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.307 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "redirect", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.308 INFO:teuthology.orchestra.run.smithi096.stdout: "redirect" 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T21:07:45.309 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "url_prefix": { 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url_prefix", 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.310 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T21:07:45.311 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_monitoring": { 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_monitoring", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "monitor device health metrics", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.312 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.313 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.314 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.315 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.316 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "mark_out_threshold": { 2024-04-25T21:07:45.317 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mark_out_threshold", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "2419200", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.318 INFO:teuthology.orchestra.run.smithi096.stdout: "pool_name": { 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pool_name", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "device_health_metrics", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "retention_period": { 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "retention_period", 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15552000", 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.320 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long to retain device health metrics", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_frequency": { 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_frequency", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.321 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "self_heal": { 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "self_heal", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.322 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:45.323 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how frequently to wake up and check device health", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "warn_threshold": { 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "warn_threshold", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.324 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7257600", 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "diskprediction_local", 2024-04-25T21:07:45.325 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.327 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.328 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.329 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.330 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "predict_interval": { 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predict_interval", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.331 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "86400", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "predictor_model": { 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "predictor_model", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.332 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "prophetstor", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.333 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "600", 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.334 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "influx", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": false, 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "influxdb python module not found", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "batch_size": { 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "batch_size", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5000", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.335 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "database": { 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "database", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "ceph", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-25T21:07:45.336 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "hostname": { 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "hostname", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server hostname", 2024-04-25T21:07:45.337 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "30", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "5", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.338 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.339 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.340 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.341 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.342 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.343 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "password": { 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "password", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.344 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "password of InfluxDB server user", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "port": { 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "port", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "8086", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.345 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "InfluxDB server port", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "ssl": { 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ssl", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "false", 2024-04-25T21:07:45.346 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "threads": { 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threads", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:45.347 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "32", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "username": { 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "username", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.348 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "username of InfluxDB server user", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "verify_ssl": { 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "verify_ssl", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "true", 2024-04-25T21:07:45.349 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "insights", 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.350 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.351 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.352 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.353 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.354 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.355 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "iostat", 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.356 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.357 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.358 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.359 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.360 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.361 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "k8sevents", 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.362 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_event_retention_days": { 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "ceph_event_retention_days", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "7", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.363 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "config_check_secs": { 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "config_check_secs", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "10", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-25T21:07:45.364 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.365 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.366 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.367 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.369 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "localpool", 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.370 INFO:teuthology.orchestra.run.smithi096.stdout: "failure_domain": { 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "failure_domain", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "host", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "failure domain for any created local pool", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.371 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.372 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.373 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.374 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.375 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.376 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min_size": { 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "min_size", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.377 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default min_size for any created local pool", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "num_rep": { 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "num_rep", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3", 2024-04-25T21:07:45.378 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default replica count for any created local pool", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_num": { 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_num", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "128", 2024-04-25T21:07:45.379 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "default pg_num for any created local pool", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "prefix": { 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prefix", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.380 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "name prefix for any created local pool", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "subtree": { 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "subtree", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "rack", 2024-04-25T21:07:45.381 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mds_autoscaler", 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.382 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.383 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.384 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.385 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.387 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirroring", 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.388 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.389 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.390 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.391 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.392 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "nfs", 2024-04-25T21:07:45.393 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.394 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.395 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.396 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.397 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.398 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "fail_fs": { 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "fail_fs", 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.399 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.400 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.401 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.402 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.403 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.404 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator": { 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "orchestrator", 2024-04-25T21:07:45.405 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "cephadm", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "rook", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "test_orchestrator" 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Orchestrator backend", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.406 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_perf_query", 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.407 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.408 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.410 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.411 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.412 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "osd_support", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.413 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.414 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.415 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.416 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.417 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "pg_autoscaler", 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.418 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.419 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.420 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.421 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.422 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.423 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.424 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "threshold": { 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "threshold", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "3.0", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1.0", 2024-04-25T21:07:45.425 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "scaling threshold", 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "progress", 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.426 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_pg_recovery_event": { 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_pg_recovery_event", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.427 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.428 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.429 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.430 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.431 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.432 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.433 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "max_completed_events": { 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_completed_events", 2024-04-25T21:07:45.434 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "50", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "number of past completed events to remember", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "sleep_interval": { 2024-04-25T21:07:45.435 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "sleep_interval", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "5", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "how long the module is going to sleep", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "prometheus", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "cache": { 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "cache", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.437 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "exclude_perf_counters": { 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "exclude_perf_counters", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.438 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.439 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.440 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.441 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.442 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.443 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools": { 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.444 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "300", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.445 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "scrape_interval": { 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "scrape_interval", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15.0", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.446 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "::", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.447 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "9283", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.448 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "stale_cache_strategy": { 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stale_cache_strategy", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "log", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.449 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_behaviour": { 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_behaviour", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "default", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.450 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "error" 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "standby_error_status_code": { 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "standby_error_status_code", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "500", 2024-04-25T21:07:45.451 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "400", 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "599", 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.452 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.453 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.454 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.455 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.456 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.457 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_snap_create": { 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_snap_create", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.458 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "mirror_snapshot_schedule": { 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "mirror_snapshot_schedule", 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.459 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "trash_purge_schedule": { 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "trash_purge_schedule", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.460 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "restful", 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.461 INFO:teuthology.orchestra.run.smithi096.stdout: "enable_auth": { 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enable_auth", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.462 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "key_file": { 2024-04-25T21:07:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "key_file", 2024-04-25T21:07:45.465 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.466 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.467 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.469 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.471 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "server_addr": { 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_addr", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.472 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "server_port": { 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "server_port", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.473 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rgw", 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.474 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.475 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.476 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.477 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.478 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rook", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.479 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.480 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.481 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.482 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.483 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.484 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "storage_class": { 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "storage_class", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "local", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.485 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "selftest", 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.486 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.487 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.488 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.489 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.490 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.491 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "roption1": { 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption1", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.492 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "roption2": { 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "roption2", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "xyz", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.493 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption1": { 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption1", 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.494 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption2": { 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption2", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.495 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption3": { 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption3", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "float", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.496 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption4": { 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption4", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.497 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption5": { 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption5", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.498 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption6": { 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption6", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.499 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "rwoption7": { 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rwoption7", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.500 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "1", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "42", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "testkey": { 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testkey", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.501 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "testlkey": { 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testlkey", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.502 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "testnewline": { 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "testnewline", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.503 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.504 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "allow_m_granularity": { 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "allow_m_granularity", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.505 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "allow minute scheduled snapshots", 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "dump_on_update": { 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "dump_on_update", 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.506 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "dump database to debug log on update", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.507 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.508 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.509 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.510 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.512 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "stats", 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.513 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.514 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.515 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.516 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.517 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.518 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.519 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "status", 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.520 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.521 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.522 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.523 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.524 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telegraf", 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "address": { 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "address", 2024-04-25T21:07:45.525 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.526 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "15", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.527 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.528 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.529 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.530 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.531 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "telemetry", 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.532 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_basic": { 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_basic", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.533 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_crash": { 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_crash", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.534 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_device": { 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_device", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.535 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_ident": { 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_ident", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "channel_perf": { 2024-04-25T21:07:45.536 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "channel_perf", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "contact": { 2024-04-25T21:07:45.537 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "contact", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "description": { 2024-04-25T21:07:45.538 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "description", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "device_url": { 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "device_url", 2024-04-25T21:07:45.539 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.540 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enabled": { 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "enabled", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.541 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.542 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "24", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "8", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "last_opt_revision": { 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "last_opt_revision", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.543 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "1", 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard": { 2024-04-25T21:07:45.544 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.545 INFO:teuthology.orchestra.run.smithi096.stdout: "leaderboard_description": { 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "leaderboard_description", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.546 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.547 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.548 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.549 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.550 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.551 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "organization": { 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "organization", 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.553 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "proxy": { 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "proxy", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.554 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "url": { 2024-04-25T21:07:45.555 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "url", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.556 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "test_orchestrator", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.557 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.558 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.559 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.560 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.561 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.562 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.563 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.564 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.565 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.566 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.567 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max_concurrent_clones": { 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "max_concurrent_clones", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "4", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.568 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "periodic_async_work": { 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "periodic_async_work", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Periodically check for async work", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.569 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_delay": { 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_delay", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "0", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-25T21:07:45.570 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "snapshot_clone_no_wait": { 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snapshot_clone_no_wait", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "True", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-25T21:07:45.571 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix", 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "can_run": true, 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "error_string": "", 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "module_options": { 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "discovery_interval": { 2024-04-25T21:07:45.572 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "discovery_interval", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "uint", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "100", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.573 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "identifier": { 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "identifier", 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.574 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "interval": { 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "interval", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "secs", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "60", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.575 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "log_level": { 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_level", 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.576 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.577 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster": { 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster", 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.578 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.579 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_cluster_level": { 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_cluster_level", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "info", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [ 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "", 2024-04-25T21:07:45.580 INFO:teuthology.orchestra.run.smithi096.stdout: "critical", 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "debug", 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "error", 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "info", 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "warning" 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "log_to_file": { 2024-04-25T21:07:45.581 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "log_to_file", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "bool", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 1, 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "False", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.582 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_host": { 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_host", 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "", 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_port": { 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_port", 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "int", 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "10051", 2024-04-25T21:07:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "zabbix_sender": { 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "zabbix_sender", 2024-04-25T21:07:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "str", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "level": "advanced", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "flags": 0, 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "min": "", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "max": "", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "enum_allowed": [], 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "desc": "", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "long_desc": "", 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "tags": [], 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "see_also": [] 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "services": { 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "dashboard": "https://172.21.15.96:8443/" 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "always_on_modules": { 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "octopus": [ 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T21:07:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "pacific": [ 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T21:07:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "quincy": [ 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T21:07:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "reef": [ 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T21:07:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: ], 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "squid": [ 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "balancer", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "crash", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "devicehealth", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "orchestrator", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "pg_autoscaler", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "progress", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "rbd_support", 2024-04-25T21:07:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "status", 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "telemetry", 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "volumes" 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "last_failure_osd_epoch": 94, 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "active_clients": [ 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "devicehealth", 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T21:07:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2350453363 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "libcephsqlite", 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 49647542 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "rbd_support", 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2545430709 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 3299856057 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "snap_schedule", 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 950879813 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "name": "volumes", 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "addrvec": [ 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: { 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "type": "v2", 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "addr": "172.21.15.96:0", 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "nonce": 2339596136 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-04-25T21:07:45.597 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:44 smithi139 ceph-mon[33027]: from='client.15903 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:44 smithi139 ceph-mon[33027]: pgmap v312: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:44 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3958895370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:44 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1628036030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:44 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2844709997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:44 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1617761607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:07:45.598 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:44 smithi097 ceph-mon[32667]: from='client.15903 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:45.599 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:44 smithi097 ceph-mon[32667]: pgmap v312: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:45.599 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:44 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3958895370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.599 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:44 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1628036030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.599 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:44 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2844709997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.599 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:44 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1617761607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2c880a3d20 msgr2=0x7f2c880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2c880a3d20 0x7f2c880a61e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2c98031440 tx=0x7f2c98005b50 comp rx=0 tx=0).stop 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 msgr2=0x7f2c9c1a0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 0x7f2c9c1a0c20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00d770 tx=0x7f2c8c00dc30 comp rx=0 tx=0).stop 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 shutdown_connections 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f2c880a3d20 0x7f2c880a61e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c9c10c520 0x7f2c9c19a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c9c103b00 0x7f2c9c1a0c20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 --2- 172.21.15.96:0/1617761607 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f2c9c1031e0 0x7f2c9c1a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:07:45.600 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 >> 172.21.15.96:0/1617761607 conn(0x7f2c9c077f50 msgr2=0x7f2c9c108570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:07:45.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 shutdown_connections 2024-04-25T21:07:45.601 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:07:44.237+0000 7f2ca443c640 1 -- 172.21.15.96:0/1617761607 wait complete. 2024-04-25T21:07:45.607 DEBUG:tasks.check_counter:Getting stats from a 2024-04-25T21:07:45.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-25T21:07:45.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:44 smithi096 ceph-mon[29288]: from='client.15903 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-25T21:07:45.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:44 smithi096 ceph-mon[29288]: pgmap v312: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:45.608 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:44 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3958895370' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:44 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1628036030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:44 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2844709997' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-25T21:07:45.609 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:44 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1617761607' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-25T21:07:45.856 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:46.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:46 smithi096 ceph-mon[29288]: pgmap v313: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:46.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:46 smithi097 ceph-mon[32667]: pgmap v313: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:46.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:46 smithi139 ceph-mon[33027]: pgmap v313: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:47.714 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3040, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001471986, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000033588, 2024-04-25T21:07:47.715 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000446575, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000639643, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000213214 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:07:47.716 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 168, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 185776, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.014113701, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.009984913, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001777678, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 165, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.011873109, 2024-04-25T21:07:47.717 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000071958 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 185776 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 231, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 220, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1592432, 2024-04-25T21:07:47.718 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 135104, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.043005824, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.014234522, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.033577655, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004221136, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 219, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009228049, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000042137 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:47.719 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1592432, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 135104 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:07:47.720 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:07:47.721 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T21:07:47.722 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148731, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 31, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T21:07:47.723 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:07:47.724 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 233, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1559837, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 233, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 233, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1559837, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:47.725 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:47.726 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:47.726 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:07:48.333 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:48 smithi096 ceph-mon[29288]: pgmap v314: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-04-25T21:07:48.360 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2024-04-25T21:07:48.360 DEBUG:tasks.check_counter:Getting stats from d 2024-04-25T21:07:48.361 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-25T21:07:48.608 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:48.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:48 smithi097 ceph-mon[32667]: pgmap v314: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:48.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:48 smithi139 ceph-mon[33027]: pgmap v314: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:50.312 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:07:50.312 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:07:50.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 237, 2024-04-25T21:07:50.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 226, 2024-04-25T21:07:50.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1594064, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 138944, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.043728035, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.014076502, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.032933681, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004442437, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 225, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.010846575, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000048207 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:50.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1594064, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 138944 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 173, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 186656, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T21:07:50.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.013348409, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.009316477, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001823583, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 170, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.015945126, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000093794 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 186656 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3040, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.000606861, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000039291, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000555882, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:50.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000742535, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000247511 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T21:07:50.317 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:07:50.318 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:07:50.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148754, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 31, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:07:50.320 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T21:07:50.321 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 239, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1560617, 2024-04-25T21:07:50.322 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 239, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 239, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1560617, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:50.323 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:07:50.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:50 smithi096 ceph-mon[29288]: pgmap v315: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:50.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:50 smithi097 ceph-mon[32667]: pgmap v315: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:50.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:50 smithi139 ceph-mon[33027]: pgmap v315: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-25T21:07:50.960 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-04-25T21:07:50.960 DEBUG:tasks.check_counter:Getting stats from g 2024-04-25T21:07:50.961 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-25T21:07:51.199 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:52.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:52 smithi096 ceph-mon[29288]: pgmap v316: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:52.374 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:52 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:52.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:52 smithi097 ceph-mon[32667]: pgmap v316: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:52.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:52 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:52.621 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:52 smithi139 ceph-mon[33027]: pgmap v316: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:52.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:52 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-25T21:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:07:52.801 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 4, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 4, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3040, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 448, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.001703250, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000048912, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000695403, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-04-25T21:07:52.802 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000804313, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000268104 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 169, 2024-04-25T21:07:52.803 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 185952, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.013292299, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.009362605, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001803221, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 166, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.008528282, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000051375 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:52.804 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 185952 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 233, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 222, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1592976, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 136384, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.042067650, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.013756488, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.033059528, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004441390, 2024-04-25T21:07:52.805 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 221, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.010505201, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000047534 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1592976, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 136384 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T21:07:52.806 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:07:52.807 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:07:52.808 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148754, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 31, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T21:07:52.809 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T21:07:52.810 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 235, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1560097, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 235, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 235, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1560097, 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T21:07:52.811 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:52.812 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:52.812 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:52.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:52.812 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:52.812 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:07:53.572 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:53 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:53 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:53 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:53 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:53 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.573 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:53 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:53 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:53 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:53 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:53 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:53 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:53 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-04-25T21:07:53.598 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2024-04-25T21:07:53.598 DEBUG:tasks.check_counter:Getting stats from j 2024-04-25T21:07:53.599 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-25T21:07:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:53 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:53 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:53 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:53 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:53 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:53.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:53 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.516 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:07:54.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:54 smithi096 ceph-mon[29288]: pgmap v317: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:54.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:54 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.787 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:54 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.788 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:54 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.788 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:54 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:54 smithi097 ceph-mon[32667]: pgmap v317: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:54 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:54 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:54 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:54 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:54 smithi139 ceph-mon[33027]: pgmap v317: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-25T21:07:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:54 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:54 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:54 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:54.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:54 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.864 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:55 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.872 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:55 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.891 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.891 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:55.892 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:55 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:56.311 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-04-25T21:07:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:07:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 0, 2024-04-25T21:07:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2, 2024-04-25T21:07:56.311 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 0, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 0, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.000960326, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.000112290, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:56.312 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:07:56.313 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 243, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 230, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1597648, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 140672, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 2, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.043848248, 2024-04-25T21:07:56.314 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.015052984, 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.033221361, 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004588777, 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 230, 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.011199229, 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000048692 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:56.315 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1597648, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 140672 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 175, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 187008, 2024-04-25T21:07:56.316 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 5, 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.015068426, 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.010993380, 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001743839, 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:56.317 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 172, 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.012185483, 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000070845 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 187008 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 0, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:07:56.319 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:07:56.320 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:07:56.321 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:07:56.322 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-04-25T21:07:56.322 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:07:56.322 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:07:56.322 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:07:56.322 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-04-25T21:07:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-04-25T21:07:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-04-25T21:07:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:07:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:07:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:07:56.323 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 148692, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 30, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-04-25T21:07:56.324 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 928, 2024-04-25T21:07:56.325 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 2, 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:07:56.326 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 241, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 1560877, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 241, 2024-04-25T21:07:56.327 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 241, 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1560877, 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-04-25T21:07:56.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:56.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-04-25T21:07:56.329 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-04-25T21:07:56.786 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:56 smithi139 ceph-mon[33027]: pgmap v318: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:56.786 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:56 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T21:07:56.786 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:56 smithi139 ceph-mon[33027]: Unable to set osd_memory_target on smithi097 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:07:56.787 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:56 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi139 to 176.2M 2024-04-25T21:07:56.787 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:56 smithi139 ceph-mon[33027]: Unable to set osd_memory_target on smithi139 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T21:07:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:56 smithi097 ceph-mon[32667]: pgmap v318: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:56 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T21:07:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:56 smithi097 ceph-mon[32667]: Unable to set osd_memory_target on smithi097 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:07:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:56 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi139 to 176.2M 2024-04-25T21:07:56.863 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:56 smithi097 ceph-mon[32667]: Unable to set osd_memory_target on smithi139 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T21:07:56.948 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:56 smithi096 ceph-mon[29288]: pgmap v318: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-25T21:07:56.949 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:56 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi097 to 176.2M 2024-04-25T21:07:56.949 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:56 smithi096 ceph-mon[29288]: Unable to set osd_memory_target on smithi097 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:07:56.949 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:56 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi139 to 176.2M 2024-04-25T21:07:56.949 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:56 smithi096 ceph-mon[29288]: Unable to set osd_memory_target on smithi139 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-25T21:07:56.975 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-25T21:07:56.976 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-04-25T21:07:56.976 DEBUG:tasks.check_counter:Getting stats from b 2024-04-25T21:07:56.976 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-25T21:07:57.232 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: pgmap v319: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:58.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:58.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:58 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: pgmap v319: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:58.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:58 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: pgmap v319: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-25T21:07:58.546 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-25T21:07:58.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:58 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' 2024-04-25T21:07:59.143 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T21:07:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:07:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 30243, 2024-04-25T21:07:59.143 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 29441, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 7880436, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 85238115, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 55, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 1.942425387, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.732590267, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.464699180, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.623926949, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 29435, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.498997131, 2024-04-25T21:07:59.144 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000118871 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1280, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000208583, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000162 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 451008, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 121808 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 74919, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 125044, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 24681851, 2024-04-25T21:07:59.145 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 103736710, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 26, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 4.342553819, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 1.792671024, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 3.014398399, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.612448952, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 125046, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 8.627764809, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000068996 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.146 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 117103, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.017380014, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000148 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 466112 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 13233, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 14691, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 5169517, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 21352998, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 10, 2024-04-25T21:07:59.147 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T21:07:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.965447690, 2024-04-25T21:07:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.375144009, 2024-04-25T21:07:59.148 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.289312322, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.260839901, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 14690, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.740710407, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000186569 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5963, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001052907, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000176 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1725536, 2024-04-25T21:07:59.149 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 148800 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 89137, 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 27.249534987, 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000305703 2024-04-25T21:07:59.150 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5809, 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.463430889, 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000251924 2024-04-25T21:07:59.151 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 21789, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 21688, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 21688, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 143.585668149, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.006620512 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T21:07:59.152 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 101, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 1, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 702, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 22, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 10938, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 2772, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 8156, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 10, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 6978, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 1484, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 2, 2024-04-25T21:07:59.153 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 3, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 12, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 25791, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 22927, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 35, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 1, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 4, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 66, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 1429, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 66, 2024-04-25T21:07:59.154 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 4339, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 4, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 234, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 7, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T21:07:59.155 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 6, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 8798, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 17542, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 7861, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 15052, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 13130, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 447, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 13670 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 290, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 3, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 2, 2024-04-25T21:07:59.156 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 2, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 658, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 59, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 3696, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 3634, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 3, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T21:07:59.157 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 2, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 580, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 66, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 22, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 46261, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 42365, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 42365, 2024-04-25T21:07:59.158 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 3896, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 5943, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 80, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 73, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 73, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 7, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 1, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 8, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 129841354, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 141172622, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T21:07:59.159 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 25486, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 89.640497037, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003517244 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 6175, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 11242, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 5067, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 233, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 264, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 31, 2024-04-25T21:07:59.160 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 10938, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 16611, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 5673, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 3, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 5928, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 5925, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 122652, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 231800 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 40476, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 1610, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 21789, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 101, 2024-04-25T21:07:59.161 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 3240, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5053, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 72.050772900, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.014259009 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 338, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 7.715816074, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.022827858 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.162 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003719959, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001239986 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2760, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.497231916, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000542475 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T21:07:59.163 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000156705, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000156705 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T21:07:59.164 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T21:07:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.165 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001588822, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000158882 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 71, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 19.233893105, 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.270899902 2024-04-25T21:07:59.166 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.154515142, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.015451514 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 406, 2024-04-25T21:07:59.167 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.105693868, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000260329 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 334, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.001242297, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002997731 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 79, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.184303654, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002332957 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T21:07:59.168 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 65, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.732531031, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.042038938 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.169 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.014236556, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004745518 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8962, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 32.580007747, 2024-04-25T21:07:59.170 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003635350 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.010001012, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001250126 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.171 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020494490, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002561811 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019313212, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003862642 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.172 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3572, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 6.260149659, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001752561 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T21:07:59.173 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 6, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 657, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 328, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 610, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 360, 2024-04-25T21:07:59.174 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 360, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:07:59.175 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T21:07:59.176 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 1007505, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 6248, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 88, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 1, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T21:07:59.177 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7840, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 163, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 22494870, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 224240, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T21:07:59.178 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 71193, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 145794155, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 71193, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 71193, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 654.294566887, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.009190433 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T21:07:59.179 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 71193, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 79817 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 71193, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 56, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 71137, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 449, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 3641, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 55, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 25541, 2024-04-25T21:07:59.180 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 122, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 6, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 4252, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 7284, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:07:59.181 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 36715, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T21:07:59.182 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 101, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 10, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2024-04-25T21:07:59.183 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 1594, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 155 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.184 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 625 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 34, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 10, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 3663, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 3634, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:07:59.185 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 118391, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 27006615, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 118391, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 118391, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 27006615, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.186 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 71193, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 146224613, 2024-04-25T21:07:59.187 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 71193, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 146224613, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:07:59.188 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 71193, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 71193, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 71193, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 71193, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:07:59.189 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3634, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 430474, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3634, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 55, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 430474, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.190 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55ceafd20500": { 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 46261, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 136978318, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 46261, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 25486, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 136978318, 2024-04-25T21:07:59.191 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:07:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:07:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:07:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:07:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.192 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:07:59.192 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T21:07:59.445 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:59 smithi097 ceph-mon[32667]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T21:07:59.445 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:07:59 smithi097 ceph-mon[32667]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:07:59.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:59 smithi096 ceph-mon[29288]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T21:07:59.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:07:59 smithi096 ceph-mon[29288]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:07:59.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:59 smithi139 ceph-mon[33027]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-04-25T21:07:59.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:07:59 smithi139 ceph-mon[33027]: Unable to set osd_memory_target on smithi096 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-25T21:07:59.917 INFO:tasks.check_counter:Daemon mds.b mds.exported=66 2024-04-25T21:07:59.917 INFO:tasks.check_counter:Daemon mds.b mds.imported=66 2024-04-25T21:07:59.917 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=290 2024-04-25T21:07:59.917 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3 2024-04-25T21:07:59.917 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=7 2024-04-25T21:07:59.917 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2024-04-25T21:07:59.918 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=3 2024-04-25T21:07:59.918 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=22 2024-04-25T21:07:59.918 DEBUG:tasks.check_counter:Getting stats from e 2024-04-25T21:07:59.918 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-25T21:08:00.158 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:08:00.492 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:00 smithi097 ceph-mon[32667]: pgmap v320: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:00.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:00 smithi096 ceph-mon[29288]: pgmap v320: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:00.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:00 smithi139 ceph-mon[33027]: pgmap v320: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 452, 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 452, 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 4319167, 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 142976, 2024-04-25T21:08:01.814 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 2, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.042065417, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.015978661, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.011497685, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.010063933, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 451, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.017274021, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000038301 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.815 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 633, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 805, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 4351724, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 661383, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-25T21:08:01.816 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.065613347, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.023316233, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.016225165, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.019141102, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 802, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.041361346, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000051572 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.817 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 460720 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1436, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1414, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 5513257, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 515194, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-25T21:08:01.818 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.171731387, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.055841724, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.069868525, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.045006013, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1413, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.064394662, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000045573 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.819 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1660976, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 142816 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T21:08:01.820 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3833, 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.111070399, 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000028977 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.821 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T21:08:01.822 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 1506, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 1506, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 1506, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2024-04-25T21:08:01.823 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 1, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T21:08:01.824 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:08:01.825 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:08:01.826 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2024-04-25T21:08:01.827 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T21:08:01.828 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5212, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T21:08:01.829 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 8, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 16070915, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 16068354, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.830 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 5212 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 25, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 4363, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 4338, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 8, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 86, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 78, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 1506, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 5844, 2024-04-25T21:08:01.831 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 4338, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 67004, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 215416 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2024-04-25T21:08:01.832 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.833 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T21:08:01.834 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T21:08:01.835 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.836 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.837 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T21:08:01.838 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.839 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T21:08:01.840 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T21:08:01.841 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.842 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.843 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T21:08:01.844 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.845 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-25T21:08:01.846 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 3, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 3, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 2374, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:01.847 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:01.848 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:01.849 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 199724, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 78, 2024-04-25T21:08:01.850 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7840, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 163, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:01.851 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 1137658, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 6652, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2250, 2024-04-25T21:08:01.852 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 21, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 3, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2250, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2250, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1.171979582, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000520879 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2250, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2251 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.853 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2250, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2250, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1415, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 833, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T21:08:01.854 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 1, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T21:08:01.855 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T21:08:01.856 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 101, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 10, 2024-04-25T21:08:01.857 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 6, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 6, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T21:08:01.858 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 0, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.859 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 2517, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 13941008, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 2517, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2517, 2024-04-25T21:08:01.860 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 13941008, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:08:01.861 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2250, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 11880868, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2250, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 11880868, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.862 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2250, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2250, 2024-04-25T21:08:01.863 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2250, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2250, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:01.864 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.865 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55ec2590c6e0": { 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:01.866 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:01.867 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T21:08:02.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:02 smithi096 ceph-mon[29288]: pgmap v321: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:02.531 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:02 smithi097 ceph-mon[32667]: pgmap v321: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:08:02.558 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-04-25T21:08:02.558 DEBUG:tasks.check_counter:Getting stats from h 2024-04-25T21:08:02.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-25T21:08:02.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:02 smithi139 ceph-mon[33027]: pgmap v321: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-04-25T21:08:02.794 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:08:04.436 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:04 smithi097 ceph-mon[32667]: pgmap v322: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 928, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1126, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 232062, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 4689560, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 20, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 7, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.075069749, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.026712021, 2024-04-25T21:08:04.437 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.024766046, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.017936783, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1124, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.086060757, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000076566 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 628, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000086863, 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000138 2024-04-25T21:08:04.438 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 928, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 1216 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 841, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 846, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1814290, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 1668321, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 7, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.101457803, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.035853120, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.050718744, 2024-04-25T21:08:04.439 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.020664117, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 846, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.107940029, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000127588 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 59, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000018955, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000321 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1680048, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 159888 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.440 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 10007, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 15139, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 5959850, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 14327689, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 10, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.661819673, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.323322256, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.371897184, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.019337910, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 15136, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.467844582, 2024-04-25T21:08:04.441 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000030909 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 22175, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003299700, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000148 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 462160 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T21:08:04.442 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 343, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2.485020095, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.007244956 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2292, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.529408048, 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000230980 2024-04-25T21:08:04.443 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 3550, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 3247, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3247, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 4.003976593, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001233131 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 303, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2024-04-25T21:08:04.444 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 11, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 49, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 2, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 1, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 46, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 46, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 4462, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 2, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 2, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 3, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 7055, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 4886, 2024-04-25T21:08:04.445 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 301, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 125, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 4, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 65, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 4322, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 65, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 1423, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:08:04.446 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 2771, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 79087400, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 18, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 1554, 2024-04-25T21:08:04.447 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 2, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 65, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 47, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 22, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 3123 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 189, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 358, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 209, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 273, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 192, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 1, 2024-04-25T21:08:04.448 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 1468, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 1467, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_path": 2, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_quiesce_inode": 19, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 65, 2024-04-25T21:08:04.449 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "evlrg": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 5212, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 5212, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2024-04-25T21:08:04.450 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 8, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 8, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "segmjr": 2, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 16070915, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 204, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 4.508010190, 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.022098089 2024-04-25T21:08:04.451 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 36, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 4430, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 4394, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 20, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 245, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 225, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 49, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 5990, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 5941, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 2, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 2880, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 2878, 2024-04-25T21:08:04.452 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 82396, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 223608 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 5215, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 5, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 3550, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 22, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 1588, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 45, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.089867641, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001997058 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.453 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 3, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000754166, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000251388 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.454 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1568, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.703083018, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000448394 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000112375, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000112375 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.455 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.456 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T21:08:04.457 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.001849879, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001849879 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2024-04-25T21:08:04.458 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 12, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002975967, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000247997 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 133, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.095934666, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000721313 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.026751297, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002057792 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.459 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.066179836, 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.066179836 2024-04-25T21:08:04.460 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.007086301, 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003543150 2024-04-25T21:08:04.461 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.462 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000719078, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000719078 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1467, 2024-04-25T21:08:04.463 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.008662369, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002050894 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 3, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 1, 2024-04-25T21:08:04.464 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 2824, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 1412, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 599, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "mdthresh_evicted": 0 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:04.465 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:04.466 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 313975, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 131, 2024-04-25T21:08:04.467 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 7840, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 163, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 156503, 2024-04-25T21:08:04.468 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 1189, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 1798, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 12182949, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 33, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 1798, 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2024-04-25T21:08:04.469 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1798, 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 7.014877571, 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003901489 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1798, 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 1841 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.470 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 1798, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 23, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 1775, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 23, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 227, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 24, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2024-04-25T21:08:04.471 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 1481, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:08:04.472 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 2, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2024-04-25T21:08:04.473 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 101, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 10, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 44, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 0, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 10 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2024-04-25T21:08:04.474 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 21 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 10, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 10, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed_ops": 1467, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 1467, 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2024-04-25T21:08:04.475 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 11772, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 6927798, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 11772, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 11772, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 6927798, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.476 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 1798, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 12365164, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1798, 2024-04-25T21:08:04.477 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 12365164, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:04.478 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 1798, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 1798, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1798, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1798, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:04.479 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 1467, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 182215, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 1467, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 23, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 182215, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x55d1a33da500": { 2024-04-25T21:08:04.480 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 5212, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 11876611, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 5212, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 204, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 11876611, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:04.481 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.482 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:04.482 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T21:08:04.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:04 smithi096 ceph-mon[29288]: pgmap v322: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:04.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:04 smithi139 ceph-mon[33027]: pgmap v322: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds.exported=65 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds.imported=65 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=189 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=358 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:08:05.184 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-04-25T21:08:05.184 DEBUG:tasks.check_counter:Getting stats from k 2024-04-25T21:08:05.185 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-25T21:08:05.426 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.b/config 2024-04-25T21:08:06.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:06 smithi096 ceph-mon[29288]: pgmap v323: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:08:06.539 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:06 smithi097 ceph-mon[32667]: pgmap v323: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:08:06.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:06 smithi139 ceph-mon[33027]: pgmap v323: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-04-25T21:08:07.068 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.001081548, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000236236, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:07.069 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 2, 2024-04-25T21:08:07.070 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 178, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 320, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 187536, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.011304043, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.007373120, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001780243, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 175, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.015514942, 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000088656 2024-04-25T21:08:07.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 187536 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 247, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 234, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1598736, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 143232, 2024-04-25T21:08:07.072 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.046083972, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.014172653, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.035285657, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.005548750, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 234, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.011780853, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000050345 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:07.073 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1598736, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 143232 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:07.074 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:07.075 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:07.076 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 148754, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 31, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2024-04-25T21:08:07.077 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:07.078 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 245, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1561397, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 245, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 245, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1561397, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2024-04-25T21:08:07.079 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2024-04-25T21:08:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:07.080 INFO:teuthology.orchestra.run.smithi097.stdout: } 2024-04-25T21:08:07.080 INFO:teuthology.orchestra.run.smithi097.stdout:} 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-04-25T21:08:07.783 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2024-04-25T21:08:07.783 DEBUG:tasks.check_counter:Getting stats from c 2024-04-25T21:08:07.784 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-25T21:08:08.037 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:08:08.467 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:08 smithi139 ceph-mon[33027]: pgmap v324: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:08:08.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:08 smithi096 ceph-mon[29288]: pgmap v324: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:08:08.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:08 smithi097 ceph-mon[32667]: pgmap v324: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 846, 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 837, 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 46583142, 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 301174, 2024-04-25T21:08:10.194 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 7, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.111209457, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.027893347, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.051759170, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.025156680, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 837, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.036846700, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000044022 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:10.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 225104, 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 80160 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 520, 2024-04-25T21:08:10.196 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 615, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 36745059, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 419579, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.062885555, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.018717975, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.025703317, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.014077310, 2024-04-25T21:08:10.197 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 613, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.021287126, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000034726 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T21:08:10.198 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 254480 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 460, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 462, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 32247041, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 146296, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.050554130, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.014912153, 2024-04-25T21:08:10.199 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.021646891, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.010343135, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 460, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.017834132, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000038769 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.200 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2742, 2024-04-25T21:08:10.201 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.093369890, 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000034051 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-04-25T21:08:10.203 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 14498, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 4393, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 10105, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 8223, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-04-25T21:08:10.204 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 2, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-04-25T21:08:10.205 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 4, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 234, 2024-04-25T21:08:10.206 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 7, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:08:10.207 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.208 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-04-25T21:08:10.209 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:08:10.210 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:08:10.211 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-04-25T21:08:10.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-04-25T21:08:10.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:08:10.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-04-25T21:08:10.214 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-04-25T21:08:10.214 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.214 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 36638, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 7, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 13, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-04-25T21:08:10.215 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 129841354, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 141178892, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 141178832, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 36638 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 6169, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 9781, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 3612, 2024-04-25T21:08:10.216 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 225, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 230, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 5, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 14498, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 14510, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 12, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 80824, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215392 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-04-25T21:08:10.217 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.218 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.219 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.222 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-04-25T21:08:10.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.223 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.224 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.225 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.226 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-04-25T21:08:10.227 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.228 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-04-25T21:08:10.229 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 2, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 1804, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:10.230 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:10.231 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-04-25T21:08:10.232 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 460914, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 6319, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 7840, 2024-04-25T21:08:10.233 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 163, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 24736046, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 208780, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-04-25T21:08:10.234 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 1674, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 26, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 4, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 1674, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1674, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.128975368, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000674417 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1674, 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1675 2024-04-25T21:08:10.235 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 1674, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 1674, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 999, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 672, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-04-25T21:08:10.236 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 2, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:08:10.237 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-04-25T21:08:10.238 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 102, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 7, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 10, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 10, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-04-25T21:08:10.239 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-04-25T21:08:10.240 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 1822, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 115400711, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 1822, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:10.241 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1822, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 115400711, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:10.242 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 1674, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 114882008, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1674, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 114882008, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.243 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 1674, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 1674, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1674, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 1674, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.244 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:10.245 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x562dfc16a6e0": { 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:10.246 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:10.247 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T21:08:10.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:10 smithi096 ceph-mon[29288]: pgmap v325: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:08:10.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:10 smithi096 ceph-mon[29288]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:08:10.529 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:10 smithi139 ceph-mon[33027]: pgmap v325: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:08:10.529 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:10 smithi139 ceph-mon[33027]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:08:10.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:10 smithi097 ceph-mon[32667]: pgmap v325: 97 pgs: 97 active+clean; 84 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-04-25T21:08:10.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:10 smithi097 ceph-mon[32667]: osdmap e102: 12 total, 12 up, 12 in 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=7 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:08:11.033 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-04-25T21:08:11.033 DEBUG:tasks.check_counter:Getting stats from f 2024-04-25T21:08:11.034 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-25T21:08:11.272 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:08:12.453 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:12 smithi139 ceph-mon[33027]: pgmap v327: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:08:12.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:12 smithi096 ceph-mon[29288]: pgmap v327: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:08:12.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:12 smithi097 ceph-mon[32667]: pgmap v327: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:08:13.118 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T21:08:13.118 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:13.118 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 2363, 2024-04-25T21:08:13.118 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2363, 2024-04-25T21:08:13.118 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 737685, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 750617, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 2, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.201469394, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.064982371, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.053129231, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.068340417, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2362, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.084084456, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000035598 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.119 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 448 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 4, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 178, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 876, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 465002, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 6, 2024-04-25T21:08:13.120 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 4, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.010334894, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.006554577, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.001913400, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000057584, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 175, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.013984652, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000079912 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.121 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 464304 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 269, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 247, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1662652, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 144590, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 3, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 2, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.049332591, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.014842088, 2024-04-25T21:08:13.122 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.039751546, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.005496208, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 246, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.012944217, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000052618 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1662384, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 144272 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.123 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3993, 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.107692462, 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000026970 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-04-25T21:08:13.124 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 0, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 0, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.125 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 9, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 9, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 0, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 9, 2024-04-25T21:08:13.126 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 1, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 0, 2024-04-25T21:08:13.127 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:08:13.128 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 0 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 0, 2024-04-25T21:08:13.129 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:08:13.130 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-04-25T21:08:13.131 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 44, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 2, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 2, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 4262576, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 4259971, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.132 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 44 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 5, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 31, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 26, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 3, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 15, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 12, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 9, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 38, 2024-04-25T21:08:13.133 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 29, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 50460, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 215416 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 0, 2024-04-25T21:08:13.134 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.135 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.136 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.137 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-04-25T21:08:13.138 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.139 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.140 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.141 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.142 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.143 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.144 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-04-25T21:08:13.145 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 5, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 3, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 4, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 0, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 0, 2024-04-25T21:08:13.146 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 2986, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:13.147 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:13.148 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 182158, 2024-04-25T21:08:13.149 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 50, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 7960, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 166, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:13.150 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 23862, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 197, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 2362, 2024-04-25T21:08:13.151 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 21, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 3, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 2362, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2362, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 1.153974887, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000488558 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2362, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 2363 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 2362, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 2362, 2024-04-25T21:08:13.152 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 1529, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 831, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-04-25T21:08:13.153 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 1, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-04-25T21:08:13.154 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 102, 2024-04-25T21:08:13.155 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 11, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 4, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 4, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 2, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 0 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.156 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 0 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:13.157 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 2632, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 2147568, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 2632, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2632, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 2147568, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.158 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 2362, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 68277, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2362, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 68277, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.159 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 2362, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 2362, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 2362, 2024-04-25T21:08:13.160 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 2362, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:13.161 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x5579ff2446e0": { 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:13.162 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:13.163 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:08:13.994 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-04-25T21:08:13.994 DEBUG:tasks.check_counter:Getting stats from i 2024-04-25T21:08:13.995 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-25T21:08:14.236 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:08:14.358 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:14 smithi139 ceph-mon[33027]: pgmap v328: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:08:14.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:14 smithi096 ceph-mon[29288]: pgmap v328: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:08:14.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:14 smithi097 ceph-mon[32667]: pgmap v328: 97 pgs: 40 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 37 active+clean; 80 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-04-25T21:08:16.176 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T21:08:16.176 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:16.176 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 408, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 578, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 107584, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 553632, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 17, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.049451653, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.018984308, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.031570446, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.001510557, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 572, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.035394699, 2024-04-25T21:08:16.177 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000061878 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 719, 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000147932, 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000205 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1744, 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 318640 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:16.178 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 4546, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2761, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 4362592, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 914492, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 10, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.256621790, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.098311208, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.201825006, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.010045250, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2764, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.083500517, 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000030210 2024-04-25T21:08:16.179 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 6616, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001051100, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000158 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1681728, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 153632 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 50, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 112, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 16058, 2024-04-25T21:08:16.180 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 178999, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 10, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 5, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.012065047, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.003647717, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.006623434, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000872549, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 110, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.012161827, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000110562 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 56, 2024-04-25T21:08:16.181 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000009491, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000169 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 7904, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 160560 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 1, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-MDSRank": { 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-04-25T21:08:16.182 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 108, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.012399631, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000114811 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "finisher-PurgeQueue": { 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "queue_len": 0, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "complete_latency": { 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 22, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000768064, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000034912 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.183 INFO:teuthology.orchestra.run.smithi139.stdout: "mds": { 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "request": 327, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "reply": 37, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "reply_latency": { 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 37, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.036996711, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000999911 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_reply": 0, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "forward": 290, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_complete": 0, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_fetch_keys": 0, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_commit": 11, 2024-04-25T21:08:16.184 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_split": 0, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_merge": 0, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes": 23, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_top": 4, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_bottom": 8, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pin_tail": 11, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_pinned": 14, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_expired": 136, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "inodes_with_caps": 0, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "caps": 0, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "subtrees": 4, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse": 774, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_hit": 384, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_forward": 288, 2024-04-25T21:08:16.185 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_discover": 65, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_dir_fetch": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_remote_ino": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "traverse_lock": 3, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "load_cent": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "q": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "exported": 1, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "exported_inodes": 17, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "imported": 1, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "imported_inodes": 6, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_dir_fetch": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_backtrace_fetch": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "openino_peer_discover": 0, 2024-04-25T21:08:16.186 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rfiles": 2771, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rbytes": 79087400, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "root_rsnaps": 6, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_set_tag": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_inotable_repaired": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_inodes": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "scrub_file_inodes": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_inode_file_caps": 0, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_revoke": 8, 2024-04-25T21:08:16.187 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_grant": 15, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps": 13, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_caps_dirty": 3, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_cap_release": 4, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "process_request_cap_release": 22 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_cache": { 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update": 2, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_update_receipt": 725, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_try_discover": 395, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_send_discover": 577, 2024-04-25T21:08:16.188 INFO:teuthology.orchestra.run.smithi139.stdout: "dir_handle_discover": 2, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays": 1, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_delayed": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_strays_enqueuing": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_created": 8, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_enqueued": 7, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_reintegrated": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "strays_migrated": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_processing": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_enqueued": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "num_recovering_prioritized": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_started": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "recovery_completed": 0, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_path": 2, 2024-04-25T21:08:16.189 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_quiesce_inode": 19, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_exportdir": 1, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_flush": 0, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragmentdir": 0, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_fragstats": 0, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "ireq_inodestats": 0 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_log": { 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "evlrg": 0, 2024-04-25T21:08:16.190 INFO:teuthology.orchestra.run.smithi139.stdout: "evadd": 44, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "evex": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "evtrm": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "ev": 44, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "evexg": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "evexd": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "segadd": 2, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "segex": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "segtrm": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "seg": 2, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "segmjr": 2, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "segexg": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "segexd": 0, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "expos": 4194304, 2024-04-25T21:08:16.191 INFO:teuthology.orchestra.run.smithi139.stdout: "wrpos": 4262576, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "rdpos": 0, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "jlat": { 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 25, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.057021057, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002280842 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "replayed": 0 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_mem": { 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "ino": 28, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "ino+": 107, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "ino-": 79, 2024-04-25T21:08:16.192 INFO:teuthology.orchestra.run.smithi139.stdout: "dir": 27, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dir+": 365, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dir-": 338, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dn": 23, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dn+": 171, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dn-": 148, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "cap": 0, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "cap+": 22, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "cap-": 22, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "rss": 53964, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "heap": 223608 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_server": { 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_client_request": 351, 2024-04-25T21:08:16.193 INFO:teuthology.orchestra.run.smithi139.stdout: "dispatch_server_request": 5, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_request": 327, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_client_session": 30, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "handle_peer_request": 42, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "req_create_latency": { 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 2, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.007456759, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.003728379 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getattr_latency": { 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001371632, 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000457210 2024-04-25T21:08:16.194 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getfilelock_latency": { 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "req_link_latency": { 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookup_latency": { 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 19, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.007079964, 2024-04-25T21:08:16.195 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000372629 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookuphash_latency": { 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupino_latency": { 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupname_latency": { 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.196 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupparent_latency": { 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lookupsnap_latency": { 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "req_lssnap_latency": { 2024-04-25T21:08:16.197 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mkdir_latency": { 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002086792, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002086792 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mknod_latency": { 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.198 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "req_mksnap_latency": { 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "req_open_latency": { 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000164900, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000164900 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "req_readdir_latency": { 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 3, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.001980607, 2024-04-25T21:08:16.199 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000660202 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rename_latency": { 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "req_renamesnap_latency": { 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "req_snapdiff_latency": { 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.200 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmdir_latency": { 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 1, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.002096155, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002096155 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmsnap_latency": { 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "req_rmxattr_latency": { 2024-04-25T21:08:16.201 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setattr_latency": { 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setdirlayout_latency": { 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.202 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setfilelock_latency": { 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setlayout_latency": { 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "req_setxattr_latency": { 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.203 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "req_symlink_latency": { 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "req_unlink_latency": { 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 7, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.014759902, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.002108557 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_revoke_eviction": 0, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "cap_acquisition_throttle": 0, 2024-04-25T21:08:16.204 INFO:teuthology.orchestra.run.smithi139.stdout: "req_getvxattr_latency": { 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_sessions": { 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "session_count": 2, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "session_add": 5, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "session_remove": 3, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_open": 2, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "sessions_stale": 0, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "total_load": 19, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "average_load": 9, 2024-04-25T21:08:16.205 INFO:teuthology.orchestra.run.smithi139.stdout: "avg_session_uptime": 625, 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "mdthresh_evicted": 0 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:16.206 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:16.207 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:16.208 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 271196, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 91, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-04-25T21:08:16.209 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 7960, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 166, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 125172, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 613, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:16.210 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "objecter": { 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_active": 0, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_laggy": 0, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send": 98, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_send_bytes": 82456, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_resend": 28, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_reply": 98, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "op_latency": { 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 98, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.428990693, 2024-04-25T21:08:16.211 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.004377456 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "op_inflight": 0, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "oplen_avg": { 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 98, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 134 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "op": 98, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "op_r": 1, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "op_w": 97, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "op_rmw": 0, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "op_pg": 0, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_stat": 0, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_create": 10, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_read": 1, 2024-04-25T21:08:16.212 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_write": 26, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writefull": 16, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_writesame": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_append": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_zero": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_truncate": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_delete": 19, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_mapext": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_sparse_read": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_clonerange": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_getxattr": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_setxattr": 20, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_cmpxattr": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_rmxattr": 0, 2024-04-25T21:08:16.213 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_resetxattrs": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_call": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_watch": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_notify": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_pgls_filter": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "osdop_other": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_active": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_send": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_resend": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "linger_ping": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_active": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_send": 0, 2024-04-25T21:08:16.214 INFO:teuthology.orchestra.run.smithi139.stdout: "poolop_resend": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_active": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_send": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "poolstat_resend": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_active": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_send": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "statfs_resend": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "command_active": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "command_send": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "command_resend": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "map_epoch": 103, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "map_full": 0, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "map_inc": 12, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_sessions": 10, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_open": 10, 2024-04-25T21:08:16.215 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_session_close": 0, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_laggy": 0, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_wr": 35, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_rd": 0, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_del": 7 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "oft": { 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_objs": 1, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_kv_pairs": 0, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_updates": 0, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "omap_total_removes": 0 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "osd-slow-ops": { 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "slow_ops_count": 21 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.216 INFO:teuthology.orchestra.run.smithi139.stdout: "purge_queue": { 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops": 0, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_ops_high_water": 7, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing": 0, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executing_high_water": 7, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed_ops": 7, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_executed": 7, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "pq_item_in_journal": 0 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 5000, 2024-04-25T21:08:16.217 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 4016478, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 5000, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 5000, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 4016478, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_bytes": { 2024-04-25T21:08:16.218 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 98, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 83315, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 98, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 83315, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.219 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-objecter_ops": { 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 1024, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 0, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 0, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 0, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 98, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 98, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 98, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 98, 2024-04-25T21:08:16.220 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle": { 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 7, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 859, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 7, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:16.221 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 1, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 859, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-write_buf_throttle-0x5600c59e2500": { 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 3758096384, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:16.222 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 44, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 68272, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 44, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 25, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 68272, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:16.223 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T21:08:16.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:16 smithi139 ceph-mon[33027]: pgmap v329: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:08:16.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:16 smithi139 ceph-mon[33027]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:08:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:16 smithi096 ceph-mon[29288]: pgmap v329: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:08:16.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:16 smithi096 ceph-mon[29288]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:08:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:16 smithi097 ceph-mon[32667]: pgmap v329: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2024-04-25T21:08:16.614 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:16 smithi097 ceph-mon[32667]: osdmap e103: 12 total, 12 up, 12 in 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds.exported=1 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=725 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-25T21:08:17.001 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=0 2024-04-25T21:08:17.001 DEBUG:tasks.check_counter:Getting stats from l 2024-04-25T21:08:17.001 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-25T21:08:17.239 INFO:teuthology.orchestra.run.smithi139.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.c/config 2024-04-25T21:08:18.329 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:18 smithi139 ceph-mon[33027]: pgmap v331: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:08:18.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:18 smithi096 ceph-mon[29288]: pgmap v331: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:08:18.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:18 smithi097 ceph-mon[32667]: pgmap v331: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout:{ 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 0, 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 2, 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 0, 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 0, 2024-04-25T21:08:18.950 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 4, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 0, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.001156085, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.000000000, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.000157280, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:18.951 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 6, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 182, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 3360, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 188336, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 5, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 3, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.011815464, 2024-04-25T21:08:18.952 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.007923382, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.001661053, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 180, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.019047950, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000105821 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-25T21:08:18.953 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 188336 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_messages": 247, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages": 236, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_bytes": 1596784, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_bytes": 145344, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_created_connections": 2, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_active_connections": 1, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_total_time": 0.048536963, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_send_time": 0.014996720, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_recv_time": 0.036765503, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_running_fast_dispatch_time": 0.005841782, 2024-04-25T21:08:18.954 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 235, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.013017003, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000055391 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_handle_ack_lat": { 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_recv_encrypted_bytes": 1596784, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "msgr_send_encrypted_bytes": 145344 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "cct": { 2024-04-25T21:08:18.955 INFO:teuthology.orchestra.run.smithi139.stdout: "total_workers": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "unhealthy_workers": 0 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "mempool": { 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_bytes": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bloom_filter_items": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_alloc_items": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_data_items": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T21:08:18.956 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_other_items": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_extent_items": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_blob_items": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T21:08:18.957 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_fsck_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_bytes": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_txc_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_bytes": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluestore_writing_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_bytes": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_reader_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "bluefs_file_writer_items": 0, 2024-04-25T21:08:18.958 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_bytes": 148682, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_anon_items": 29, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_bytes": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "buffer_meta_items": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_bytes": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_items": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_bytes": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_mapbl_items": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_bytes": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osd_pglog_items": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_bytes": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_items": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "osdmap_mapping_items": 0, 2024-04-25T21:08:18.959 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_bytes": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "pgmap_items": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_bytes": 928, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "mds_co_items": 2, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_bytes": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_1_items": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_bytes": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "unittest_2_items": 0 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: }, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "val": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "max": 104857600, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "get_started": 0, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "get": 249, 2024-04-25T21:08:18.960 INFO:teuthology.orchestra.run.smithi139.stdout: "get_sum": 1561917, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_fail": 0, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "get_or_fail_success": 249, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "take": 0, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "take_sum": 0, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "put": 249, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "put_sum": 1561917, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "wait": { 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "avgcount": 0, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "sum": 0.000000000, 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: "avgtime": 0.000000000 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout: } 2024-04-25T21:08:18.961 INFO:teuthology.orchestra.run.smithi139.stdout:} 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-04-25T21:08:19.664 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2024-04-25T21:08:19.665 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T21:08:19.677 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T21:08:19.677 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T21:08:19.677 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:08:19.698 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T21:08:19.698 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:08:19.758 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-25T21:08:20.045 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:20.419 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:20 smithi096 ceph-mon[29288]: pgmap v332: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 215 B/s wr, 6 op/s 2024-04-25T21:08:20.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:20 smithi097 ceph-mon[32667]: pgmap v332: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 215 B/s wr, 6 op/s 2024-04-25T21:08:20.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:20 smithi139 ceph-mon[33027]: pgmap v332: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 215 B/s wr, 6 op/s 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.741+0000 7fb244376700 1 -- 172.21.15.96:0/1209989471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 msgr2=0x7fb23c10e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.741+0000 7fb244376700 1 --2- 172.21.15.96:0/1209989471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c10e830 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fb23400b0a0 tx=0x7fb23400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- 172.21.15.96:0/1209989471 shutdown_connections 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 --2- 172.21.15.96:0/1209989471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c10e830 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 --2- 172.21.15.96:0/1209989471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23c107960 0x7fb23c107d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 --2- 172.21.15.96:0/1209989471 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb23c101960 0x7fb23c101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- 172.21.15.96:0/1209989471 >> 172.21.15.96:0/1209989471 conn(0x7fb23c0789e0 msgr2=0x7fb23c078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- 172.21.15.96:0/1209989471 shutdown_connections 2024-04-25T21:08:21.744 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- 172.21.15.96:0/1209989471 wait complete. 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 Processor -- start 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- start start 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb23c101960 0x7fb23c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23c107960 0x7fb23c1a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111ba0 con 0x7fb23c102320 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111a20 con 0x7fb23c107960 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.742+0000 7fb244376700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb23c111d20 con 0x7fb23c101960 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c19a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44376/0 (socket says 172.21.15.96:44376) 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 -- 172.21.15.96:0/1937121376 learned_addr learned my addr 172.21.15.96:0/1937121376 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb242112700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb23c101960 0x7fb23c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb242913700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23c107960 0x7fb23c1a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 -- 172.21.15.96:0/1937121376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb23c101960 msgr2=0x7fb23c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.745 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb23c101960 0x7fb23c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 -- 172.21.15.96:0/1937121376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23c107960 msgr2=0x7fb23c1a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23c107960 0x7fb23c1a4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 -- 172.21.15.96:0/1937121376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb23c1a5480 con 0x7fb23c102320 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb241911700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c19a400 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b530 tx=0x7fb22c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c015020 con 0x7fb23c102320 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb22c011ce0 con 0x7fb23c102320 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb23c1a56b0 con 0x7fb23c102320 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb22c010d20 con 0x7fb23c102320 2024-04-25T21:08:21.746 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.743+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb23c1a5bc0 con 0x7fb23c102320 2024-04-25T21:08:21.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.744+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb23c069730 con 0x7fb23c102320 2024-04-25T21:08:21.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.745+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb22c018020 con 0x7fb23c102320 2024-04-25T21:08:21.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.746+0000 7fb2337fe700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb2280a3db0 0x7fb2280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:21.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.746+0000 7fb242112700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb2280a3db0 0x7fb2280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:21.747 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.746+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8455+0+0 (secure 0 0 0) 0x7fb22c0cfef0 con 0x7fb23c102320 2024-04-25T21:08:21.748 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.746+0000 7fb242112700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb2280a3db0 0x7fb2280a6270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb238005ea0 tx=0x7fb238005e30 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:21.751 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.750+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb22c01f020 con 0x7fb23c102320 2024-04-25T21:08:21.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.965+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb23c1a5e80 con 0x7fb23c102320 2024-04-25T21:08:21.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.965+0000 7fb2337fe700 1 -- 172.21.15.96:0/1937121376 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v103) ==== 81+0+940 (secure 0 0 0) 0x7fb22c09c6e0 con 0x7fb23c102320 2024-04-25T21:08:21.965 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2385969372 2024-04-26T20:57:38.516139+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1528469114 2024-04-26T20:57:38.516139+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/23858312 2024-04-26T20:57:38.516139+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/995634760 2024-04-26T20:57:38.516139+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1163862190 2024-04-26T20:47:09.141019+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2305167259 2024-04-26T20:47:09.141019+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/1894355014 2024-04-26T20:47:09.141019+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/3915899740 2024-04-26T20:46:41.969698+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/1496220452 2024-04-26T20:46:41.969698+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/111790105 2024-04-26T20:46:41.969698+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2748921341 2024-04-26T20:46:41.969698+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6801/23858312 2024-04-26T20:57:38.516139+0000 2024-04-25T21:08:21.966 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2934738303 2024-04-26T20:57:38.516139+0000 2024-04-25T21:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/1894355014 2024-04-26T20:47:09.141019+0000 2024-04-25T21:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:0/2265596716 2024-04-26T20:47:09.141019+0000 2024-04-25T21:08:21.967 INFO:teuthology.orchestra.run.smithi096.stdout:172.21.15.96:6800/3915899740 2024-04-26T20:46:41.969698+0000 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb2280a3db0 msgr2=0x7fb2280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb2280a3db0 0x7fb2280a6270 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fb238005ea0 tx=0x7fb238005e30 comp rx=0 tx=0).stop 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 msgr2=0x7fb23c19a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c19a400 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00b530 tx=0x7fb22c00b8f0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 shutdown_connections 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb2280a3db0 0x7fb2280a6270 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb23c107960 0x7fb23c1a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb23c101960 0x7fb23c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 --2- 172.21.15.96:0/1937121376 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb23c102320 0x7fb23c19a400 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:21.969 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 >> 172.21.15.96:0/1937121376 conn(0x7fb23c0789e0 msgr2=0x7fb23c1043f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 shutdown_connections 2024-04-25T21:08:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:21.968+0000 7fb244376700 1 -- 172.21.15.96:0/1937121376 wait complete. 2024-04-25T21:08:21.970 INFO:teuthology.orchestra.run.smithi096.stderr:listed 16 entries 2024-04-25T21:08:22.498 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:22 smithi096 ceph-mon[29288]: pgmap v333: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T21:08:22.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:22 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1937121376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:08:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:22 smithi097 ceph-mon[32667]: pgmap v333: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T21:08:22.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:22 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1937121376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:08:22.614 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T21:08:22.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:22.614 DEBUG:teuthology.orchestra.run.smithi096:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T21:08:22.621 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:22 smithi139 ceph-mon[33027]: pgmap v333: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T21:08:22.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:22 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1937121376' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-25T21:08:22.757 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T21:08:22.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:22.757 DEBUG:teuthology.orchestra.run.smithi096:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T21:08:22.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:22.782 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T21:08:22.859 INFO:teuthology.orchestra.run.smithi096.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T21:08:22.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:22.859 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T21:08:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:22.929 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T21:08:23.024 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T21:08:23.025 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-04-25T21:08:23.025 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T21:08:23.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=20547 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/bf1d238d3f99e53b133b81fc2dc247c3069c806ed19c39bc3b18ff494d0073a7/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/bf1d238d3f99e53b133b81fc2dc247c3069c806ed19c39bc3b18ff494d0073a7/diff,workdir=/var/lib/containers/storage/overlay/bf1d238d3f99e53b133b81fc2dc247c3069c806ed19c39bc3b18ff494d0073a7/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/a9ac87554dad2126e50f74505196b3b14729411cde621f062b8026b6a2b27a33/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/a9ac87554dad2126e50f74505196b3b14729411cde621f062b8026b6a2b27a33/diff,workdir=/var/lib/containers/storage/overlay/a9ac87554dad2126e50f74505196b3b14729411cde621f062b8026b6a2b27a33/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/4eb8f7ae9002b6ab83ed055429d22834fb851e2683d8db1a96bed85824ad8b31/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/4eb8f7ae9002b6ab83ed055429d22834fb851e2683d8db1a96bed85824ad8b31/diff,workdir=/var/lib/containers/storage/overlay/4eb8f7ae9002b6ab83ed055429d22834fb851e2683d8db1a96bed85824ad8b31/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/7fc45437281745e226c1ad6b772922833a923936653845b66f7da7d7f071c878/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/7fc45437281745e226c1ad6b772922833a923936653845b66f7da7d7f071c878/diff,workdir=/var/lib/containers/storage/overlay/7fc45437281745e226c1ad6b772922833a923936653845b66f7da7d7f071c878/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/385399ee2dd21cb9e27c938920b94ea63d2384b948d6c7adc06e7b2a0e20b4b3/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/385399ee2dd21cb9e27c938920b94ea63d2384b948d6c7adc06e7b2a0e20b4b3/diff,workdir=/var/lib/containers/storage/overlay/385399ee2dd21cb9e27c938920b94ea63d2384b948d6c7adc06e7b2a0e20b4b3/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/3bc812c94327a1b10c0366fba2b01a0e33fd8f0c9f25006439f9377a7261a593/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/3bc812c94327a1b10c0366fba2b01a0e33fd8f0c9f25006439f9377a7261a593/diff,workdir=/var/lib/containers/storage/overlay/3bc812c94327a1b10c0366fba2b01a0e33fd8f0c9f25006439f9377a7261a593/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/be78891c47eb9f3150e7cf9704d604920c43f235e02bdd2793f4b8cb5d475e52/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/be78891c47eb9f3150e7cf9704d604920c43f235e02bdd2793f4b8cb5d475e52/diff,workdir=/var/lib/containers/storage/overlay/be78891c47eb9f3150e7cf9704d604920c43f235e02bdd2793f4b8cb5d475e52/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/56af77d809be48806571253d9d1cb66edd2b7332e9d5630a749b3223cd5d06c6/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/56af77d809be48806571253d9d1cb66edd2b7332e9d5630a749b3223cd5d06c6/diff,workdir=/var/lib/containers/storage/overlay/56af77d809be48806571253d9d1cb66edd2b7332e9d5630a749b3223cd5d06c6/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/d6747a44ff5b4298d6735b4b277c3a635e35754870d412fe1fdb0518730154e0/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/d6747a44ff5b4298d6735b4b277c3a635e35754870d412fe1fdb0518730154e0/diff,workdir=/var/lib/containers/storage/overlay/d6747a44ff5b4298d6735b4b277c3a635e35754870d412fe1fdb0518730154e0/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/58d32694929565b9b52f7cef093a863d66fc6bf3ede87aa68eecb248c99a799c/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QTWMXFXLL6RZLWF6Q36YNUKLU4:/var/lib/containers/storage/overlay/l/F27TS4WSYR5DJYAG62JG2IAKMH,upperdir=/var/lib/containers/storage/overlay/58d32694929565b9b52f7cef093a863d66fc6bf3ede87aa68eecb248c99a799c/diff,workdir=/var/lib/containers/storage/overlay/58d32694929565b9b52f7cef093a863d66fc6bf3ede87aa68eecb248c99a799c/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-25T21:08:23.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:23.042 DEBUG:teuthology.orchestra.run.smithi096:> ip netns list 2024-04-25T21:08:23.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T21:08:23.097 DEBUG:teuthology.orchestra.run.smithi096:> sudo ip link delete ceph-brx 2024-04-25T21:08:23.182 INFO:teuthology.orchestra.run.smithi096.stderr:Cannot find device "ceph-brx" 2024-04-25T21:08:23.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:08:23.184 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-25T21:08:23.195 INFO:tasks.cephadm:Teardown begin 2024-04-25T21:08:23.195 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T21:08:23.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:08:23.443 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:24.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:24 smithi096 ceph-mon[29288]: pgmap v334: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T21:08:24.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:24 smithi097 ceph-mon[32667]: pgmap v334: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T21:08:24.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:24 smithi139 ceph-mon[33027]: pgmap v334: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 102 B/s wr, 5 op/s 2024-04-25T21:08:25.189 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 -- 172.21.15.96:0/2493906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 msgr2=0x7fb5cc06b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/2493906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc06b5f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8009990 tx=0x7fb5b8009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 -- 172.21.15.96:0/2493906117 shutdown_connections 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/2493906117 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5cc06bb30 0x7fb5cc10f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/2493906117 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc108a30 0x7fb5cc108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/2493906117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc06b5f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 -- 172.21.15.96:0/2493906117 >> 172.21.15.96:0/2493906117 conn(0x7fb5cc078910 msgr2=0x7fb5cc078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 -- 172.21.15.96:0/2493906117 shutdown_connections 2024-04-25T21:08:25.190 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.189+0000 7fb5d2c89700 1 -- 172.21.15.96:0/2493906117 wait complete. 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 Processor -- start 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 -- start start 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc1990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5cc06bb30 0x7fb5cc199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc108a30 0x7fb5cc19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc112c10 con 0x7fb5cc06b190 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc112a90 con 0x7fb5cc108a30 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d2c89700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fb5cc112d90 con 0x7fb5cc06bb30 2024-04-25T21:08:25.192 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc1990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc1990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:44396/0 (socket says 172.21.15.96:44396) 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 -- 172.21.15.96:0/3706276316 learned_addr learned my addr 172.21.15.96:0/3706276316 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5cbfff700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5cc06bb30 0x7fb5cc199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 -- 172.21.15.96:0/3706276316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5cc06bb30 msgr2=0x7fb5cc199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5cc06bb30 0x7fb5cc199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 -- 172.21.15.96:0/3706276316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc108a30 msgr2=0x7fb5cc19d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc108a30 0x7fb5cc19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.190+0000 7fb5d0a25700 1 -- 172.21.15.96:0/3706276316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5cc19e0d0 con 0x7fb5cc06b190 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.191+0000 7fb5d0a25700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc1990c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000d950 tx=0x7fb5c000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.191+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c0016070 con 0x7fb5cc06b190 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.191+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5c0010460 con 0x7fb5cc06b190 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.191+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5cc19e3c0 con 0x7fb5cc06b190 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.191+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5c000f620 con 0x7fb5cc06b190 2024-04-25T21:08:25.193 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.191+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5cc101970 con 0x7fb5cc06b190 2024-04-25T21:08:25.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.192+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5cc069730 con 0x7fb5cc06b190 2024-04-25T21:08:25.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.193+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5c0019070 con 0x7fb5cc06b190 2024-04-25T21:08:25.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.193+0000 7fb5c9ffb700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb5bc0a3db0 0x7fb5bc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:25.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.194+0000 7fb5cbfff700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb5bc0a3db0 0x7fb5bc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:25.194 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.194+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8455+0+0 (secure 0 0 0) 0x7fb5c00d03c0 con 0x7fb5cc06b190 2024-04-25T21:08:25.195 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.194+0000 7fb5cbfff700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb5bc0a3db0 0x7fb5bc0a6270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b801a040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:25.198 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.198+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5c009caa0 con 0x7fb5cc06b190 2024-04-25T21:08:25.440 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.440+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5cc02df80 con 0x7fb5cc06b190 2024-04-25T21:08:25.441 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.441+0000 7fb5c9ffb700 1 -- 172.21.15.96:0/3706276316 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11865 (secure 0 0 0) 0x7fb5c009c3a0 con 0x7fb5cc06b190 2024-04-25T21:08:25.441 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:08:25.442 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T21:06:25.682066+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25093":{"gid":25093,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2108619178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2108619178},{"type":"v1","addr":"172.21.15.139:6837","nonce":2108619178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb5bc0a3db0 msgr2=0x7fb5bc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb5bc0a3db0 0x7fb5bc0a6270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb5b8006010 tx=0x7fb5b801a040 comp rx=0 tx=0).stop 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 msgr2=0x7fb5cc1990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc1990c0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fb5c000d950 tx=0x7fb5c000dd10 comp rx=0 tx=0).stop 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 shutdown_connections 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fb5bc0a3db0 0x7fb5bc0a6270 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.443+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb5cc108a30 0x7fb5cc19d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.444+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5cc06b190 0x7fb5cc1990c0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.444 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.444+0000 7fb5d2c89700 1 --2- 172.21.15.96:0/3706276316 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fb5cc06bb30 0x7fb5cc199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:25.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.444+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 >> 172.21.15.96:0/3706276316 conn(0x7fb5cc078910 msgr2=0x7fb5cc0ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:25.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.444+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 shutdown_connections 2024-04-25T21:08:25.445 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:25.444+0000 7fb5d2c89700 1 -- 172.21.15.96:0/3706276316 wait complete. 2024-04-25T21:08:25.445 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-04-25T21:08:26.086 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-25T21:08:26.325 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:26.448 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:26 smithi096 ceph-mon[29288]: pgmap v335: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-25T21:08:26.448 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:26 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3706276316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:26 smithi097 ceph-mon[32667]: pgmap v335: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-25T21:08:26.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:26 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3706276316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:26 smithi139 ceph-mon[33027]: pgmap v335: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 921 B/s wr, 7 op/s 2024-04-25T21:08:26.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:26 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3706276316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:27.970 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 -- 172.21.15.96:0/1073136863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e81034b0 msgr2=0x7fc3e810f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 --2- 172.21.15.96:0/1073136863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e81034b0 0x7fc3e810f8e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009990 tx=0x7fc3e0009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 -- 172.21.15.96:0/1073136863 shutdown_connections 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 --2- 172.21.15.96:0/1073136863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e81034b0 0x7fc3e810f8e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 --2- 172.21.15.96:0/1073136863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e8108a80 0x7fc3e8108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 --2- 172.21.15.96:0/1073136863 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e8102af0 0x7fc3e8102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 -- 172.21.15.96:0/1073136863 >> 172.21.15.96:0/1073136863 conn(0x7fc3e8079ad0 msgr2=0x7fc3e8079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 -- 172.21.15.96:0/1073136863 shutdown_connections 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.968+0000 7fc3eff17700 1 -- 172.21.15.96:0/1073136863 wait complete. 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 Processor -- start 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 -- start start 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 0x7fc3e81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e81034b0 0x7fc3e8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:27.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e8108a80 0x7fc3e819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8112c50 con 0x7fc3e8102af0 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8112ad0 con 0x7fc3e8108a80 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3eff17700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7fc3e8112dd0 con 0x7fc3e81034b0 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3edcb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 0x7fc3e81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3edcb3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 0x7fc3e81990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47446/0 (socket says 172.21.15.96:47446) 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3edcb3700 1 -- 172.21.15.96:0/3420619971 learned_addr learned my addr 172.21.15.96:0/3420619971 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.969+0000 7fc3ed4b2700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e81034b0 0x7fc3e8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3edcb3700 1 -- 172.21.15.96:0/3420619971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e81034b0 msgr2=0x7fc3e8199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3edcb3700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e81034b0 0x7fc3e8199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3ee4b4700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e8108a80 0x7fc3e819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3edcb3700 1 -- 172.21.15.96:0/3420619971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e8108a80 msgr2=0x7fc3e819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:27.972 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3edcb3700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e8108a80 0x7fc3e819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3edcb3700 1 -- 172.21.15.96:0/3420619971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3e819e0a0 con 0x7fc3e8102af0 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3ed4b2700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e81034b0 0x7fc3e8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3edcb3700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 0x7fc3e81990e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8007e20 tx=0x7fc3d800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8016020 con 0x7fc3e8102af0 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3d800ae00 con 0x7fc3e8102af0 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3e819e390 con 0x7fc3e8102af0 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3d8011700 con 0x7fc3e8102af0 2024-04-25T21:08:27.973 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.970+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3e81a5c10 con 0x7fc3e8102af0 2024-04-25T21:08:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.971+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3e8069730 con 0x7fc3e8102af0 2024-04-25T21:08:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.972+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3d8019020 con 0x7fc3e8102af0 2024-04-25T21:08:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.973+0000 7fc3e6ffd700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc3d00a3db0 0x7fc3d00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.973+0000 7fc3ed4b2700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc3d00a3db0 0x7fc3d00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.973+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8455+0+0 (secure 0 0 0) 0x7fc3d80d07e0 con 0x7fc3e8102af0 2024-04-25T21:08:27.974 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.973+0000 7fc3ed4b2700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc3d00a3db0 0x7fc3d00a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3e807a0f0 tx=0x7fc3dc00b2c0 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:27.977 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:27.977+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3d80182c0 con 0x7fc3e8102af0 2024-04-25T21:08:28.223 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.223+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3e819a420 con 0x7fc3e8102af0 2024-04-25T21:08:28.224 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.223+0000 7fc3e6ffd700 1 -- 172.21.15.96:0/3420619971 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+11865 (secure 0 0 0) 0x7fc3e819a420 con 0x7fc3e8102af0 2024-04-25T21:08:28.224 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:08:28.224 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2040913717","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2040913717},{"type":"v1","addr":"172.21.15.96:6835","nonce":2040913717}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/2090633506","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":2090633506},{"type":"v1","addr":"172.21.15.96:6837","nonce":2090633506}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/2508705262","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":2508705262},{"type":"v1","addr":"172.21.15.96:6839","nonce":2508705262}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6841/3884645906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6840","nonce":3884645906},{"type":"v1","addr":"172.21.15.96:6841","nonce":3884645906}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24383,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/2453677033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":2453677033},{"type":"v1","addr":"172.21.15.97:6833","nonce":2453677033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.139:6833/3174744460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6832","nonce":3174744460},{"type":"v1","addr":"172.21.15.139:6833","nonce":3174744460}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":35,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T20:54:31.232491+0000","modified":"2024-04-25T21:06:25.682066+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24443,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24398":{"gid":24398,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6835/122204709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":122204709},{"type":"v1","addr":"172.21.15.97:6835","nonce":122204709}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6835/1172572610","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6834","nonce":1172572610},{"type":"v1","addr":"172.21.15.139:6835","nonce":1172572610}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6837/16128615","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":16128615},{"type":"v1","addr":"172.21.15.97:6837","nonce":16128615}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.139:6839/2573809872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6838","nonce":2573809872},{"type":"v1","addr":"172.21.15.139:6839","nonce":2573809872}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.97:6839/3461334961","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3461334961},{"type":"v1","addr":"172.21.15.97:6839","nonce":3461334961}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25093":{"gid":25093,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.139:6837/2108619178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6836","nonce":2108619178},{"type":"v1","addr":"172.21.15.139:6837","nonce":2108619178}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24443,"qdb_cluster":[24442,24422,24443]},"id":1}]} 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc3d00a3db0 msgr2=0x7fc3d00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc3d00a3db0 0x7fc3d00a6270 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc3e807a0f0 tx=0x7fc3dc00b2c0 comp rx=0 tx=0).stop 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 msgr2=0x7fc3e81990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 0x7fc3e81990e0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc3d8007e20 tx=0x7fc3d800a340 comp rx=0 tx=0).stop 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 shutdown_connections 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7fc3d00a3db0 0x7fc3d00a6270 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3e8108a80 0x7fc3e819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc3e8102af0 0x7fc3e81990e0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:28.226 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 --2- 172.21.15.96:0/3420619971 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7fc3e81034b0 0x7fc3e8199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:28.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 >> 172.21.15.96:0/3420619971 conn(0x7fc3e8079ad0 msgr2=0x7fc3e8101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:28.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 shutdown_connections 2024-04-25T21:08:28.227 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:28.226+0000 7fc3eff17700 1 -- 172.21.15.96:0/3420619971 wait complete. 2024-04-25T21:08:28.227 INFO:teuthology.orchestra.run.smithi096.stderr:dumped fsmap epoch 35 2024-04-25T21:08:28.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:28 smithi096 ceph-mon[29288]: pgmap v336: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 802 B/s wr, 6 op/s 2024-04-25T21:08:28.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:28 smithi097 ceph-mon[32667]: pgmap v336: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 802 B/s wr, 6 op/s 2024-04-25T21:08:28.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:28 smithi139 ceph-mon[33027]: pgmap v336: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 802 B/s wr, 6 op/s 2024-04-25T21:08:28.879 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-25T21:08:29.119 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:29.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:29 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3420619971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:29.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:29 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3420619971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:29.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:29 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3420619971' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-25T21:08:30.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:30 smithi096 ceph-mon[29288]: pgmap v337: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:08:30.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:30 smithi097 ceph-mon[32667]: pgmap v337: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:08:30.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:30 smithi139 ceph-mon[33027]: pgmap v337: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:08:31.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 -- 172.21.15.96:0/3764160256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 msgr2=0x7f8338108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:31.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 --2- 172.21.15.96:0/3764160256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f8338108e10 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8334009990 tx=0x7f8334009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 -- 172.21.15.96:0/3764160256 shutdown_connections 2024-04-25T21:08:31.009 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 --2- 172.21.15.96:0/3764160256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833806b0f0 0x7f833810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 --2- 172.21.15.96:0/3764160256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f8338108e10 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 --2- 172.21.15.96:0/3764160256 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f833806a750 0x7f833806abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 -- 172.21.15.96:0/3764160256 >> 172.21.15.96:0/3764160256 conn(0x7f83380788f0 msgr2=0x7f8338078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 -- 172.21.15.96:0/3764160256 shutdown_connections 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.004+0000 7f8340188700 1 -- 172.21.15.96:0/3764160256 wait complete. 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 Processor -- start 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 -- start start 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f833806a750 0x7f8338199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833806b0f0 0x7f8338199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f833819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8338112c30 con 0x7f8338108a30 2024-04-25T21:08:31.010 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f8338112ab0 con 0x7f833806b0f0 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f8340188700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f8338112db0 con 0x7f833806a750 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f833e725700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f833819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f833e725700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f833819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47458/0 (socket says 172.21.15.96:47458) 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.005+0000 7f833e725700 1 -- 172.21.15.96:0/2225722945 learned_addr learned my addr 172.21.15.96:0/2225722945 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833df24700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f833806a750 0x7f8338199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833e725700 1 -- 172.21.15.96:0/2225722945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f833806a750 msgr2=0x7f8338199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833e725700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f833806a750 0x7f8338199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833d723700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833806b0f0 0x7f8338199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833e725700 1 -- 172.21.15.96:0/2225722945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833806b0f0 msgr2=0x7f8338199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833e725700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833806b0f0 0x7f8338199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833e725700 1 -- 172.21.15.96:0/2225722945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833819e150 con 0x7f8338108a30 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f833e725700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f833819da10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f833000b970 tx=0x7f833000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:31.011 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330019070 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8330014440 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833819e440 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833000ce60 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.006+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83381a5cc0 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.007+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338069730 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.008+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f833001c070 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.009+0000 7f832effd700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f83240a3db0 0x7f83240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.009+0000 7f833df24700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f83240a3db0 0x7f83240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.009+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8455+0+0 (secure 0 0 0) 0x7f83300d0010 con 0x7f8338108a30 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.009+0000 7f833df24700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f83240a3db0 0x7f83240a6270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8334006010 tx=0x7f833401a040 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:31.012 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.013+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f833009c800 con 0x7f8338108a30 2024-04-25T21:08:31.210 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.210+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f833802df80 con 0x7f8338108a30 2024-04-25T21:08:31.211 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.212+0000 7f832effd700 1 -- 172.21.15.96:0/2225722945 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+22989 (secure 0 0 0) 0x7f833009c100 con 0x7f8338108a30 2024-04-25T21:08:31.211 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-04-25T21:08:31.212 INFO:teuthology.orchestra.run.smithi096.stdout:{"epoch":103,"fsid":"b713778a-0344-11ef-bc93-c7b262605968","created":"2024-04-25T20:46:14.467106+0000","modified":"2024-04-25T21:08:15.204402+0000","last_up_change":"2024-04-25T20:53:22.931117+0000","last_in_change":"2024-04-25T20:53:01.893372+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T20:49:37.227783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T20:54:24.897398+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"91","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T20:54:28.260817+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":16,"snap_epoch":102,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8fe35739-7b4a-4c48-980e-75a5c703beaa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6803","nonce":1267246354}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6805","nonce":1267246354}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6809","nonce":1267246354}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":1267246354},{"type":"v1","addr":"172.21.15.96:6807","nonce":1267246354}]},"public_addr":"172.21.15.96:6803/1267246354","cluster_addr":"172.21.15.96:6805/1267246354","heartbeat_back_addr":"172.21.15.96:6809/1267246354","heartbeat_front_addr":"172.21.15.96:6807/1267246354","state":["exists","up"]},{"osd":1,"uuid":"2a2a1f79-9b1b-416c-9797-2417119176d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6801","nonce":3234593794}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6803","nonce":3234593794}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6807","nonce":3234593794}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3234593794},{"type":"v1","addr":"172.21.15.97:6805","nonce":3234593794}]},"public_addr":"172.21.15.97:6801/3234593794","cluster_addr":"172.21.15.97:6803/3234593794","heartbeat_back_addr":"172.21.15.97:6807/3234593794","heartbeat_front_addr":"172.21.15.97:6805/3234593794","state":["exists","up"]},{"osd":2,"uuid":"68afacf2-e118-4883-b8b9-051a6add7851","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6800","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6801","nonce":1811270506}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6802","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6803","nonce":1811270506}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6806","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6807","nonce":1811270506}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6804","nonce":1811270506},{"type":"v1","addr":"172.21.15.139:6805","nonce":1811270506}]},"public_addr":"172.21.15.139:6801/1811270506","cluster_addr":"172.21.15.139:6803/1811270506","heartbeat_back_addr":"172.21.15.139:6807/1811270506","heartbeat_front_addr":"172.21.15.139:6805/1811270506","state":["exists","up"]},{"osd":3,"uuid":"9cf6c36f-7417-4b70-961e-a1712eb00fa1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6811","nonce":2644602517}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6813","nonce":2644602517}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6817","nonce":2644602517}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2644602517},{"type":"v1","addr":"172.21.15.96:6815","nonce":2644602517}]},"public_addr":"172.21.15.96:6811/2644602517","cluster_addr":"172.21.15.96:6813/2644602517","heartbeat_back_addr":"172.21.15.96:6817/2644602517","heartbeat_front_addr":"172.21.15.96:6815/2644602517","state":["exists","up"]},{"osd":4,"uuid":"cd60dbbf-deec-4429-b64c-dc7238e7bb49","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6809","nonce":443367186}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6811","nonce":443367186}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6815","nonce":443367186}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":443367186},{"type":"v1","addr":"172.21.15.97:6813","nonce":443367186}]},"public_addr":"172.21.15.97:6809/443367186","cluster_addr":"172.21.15.97:6811/443367186","heartbeat_back_addr":"172.21.15.97:6815/443367186","heartbeat_front_addr":"172.21.15.97:6813/443367186","state":["exists","up"]},{"osd":5,"uuid":"683e5552-4261-443b-8552-397edba9e004","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6808","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6809","nonce":2385064558}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6810","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6811","nonce":2385064558}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6814","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6815","nonce":2385064558}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6812","nonce":2385064558},{"type":"v1","addr":"172.21.15.139:6813","nonce":2385064558}]},"public_addr":"172.21.15.139:6809/2385064558","cluster_addr":"172.21.15.139:6811/2385064558","heartbeat_back_addr":"172.21.15.139:6815/2385064558","heartbeat_front_addr":"172.21.15.139:6813/2385064558","state":["exists","up"]},{"osd":6,"uuid":"9a351fac-72bd-41d4-97a0-5f8010ab0eb4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6819","nonce":262813972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6821","nonce":262813972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6825","nonce":262813972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":262813972},{"type":"v1","addr":"172.21.15.96:6823","nonce":262813972}]},"public_addr":"172.21.15.96:6819/262813972","cluster_addr":"172.21.15.96:6821/262813972","heartbeat_back_addr":"172.21.15.96:6825/262813972","heartbeat_front_addr":"172.21.15.96:6823/262813972","state":["exists","up"]},{"osd":7,"uuid":"949efe04-6338-4968-a214-31dec3068888","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6817","nonce":2284407992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6819","nonce":2284407992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6823","nonce":2284407992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":2284407992},{"type":"v1","addr":"172.21.15.97:6821","nonce":2284407992}]},"public_addr":"172.21.15.97:6817/2284407992","cluster_addr":"172.21.15.97:6819/2284407992","heartbeat_back_addr":"172.21.15.97:6823/2284407992","heartbeat_front_addr":"172.21.15.97:6821/2284407992","state":["exists","up"]},{"osd":8,"uuid":"eb49bb16-6c0d-41a7-8319-cd4e96b9a89d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6816","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6817","nonce":3100320732}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6818","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6819","nonce":3100320732}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6822","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6823","nonce":3100320732}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6820","nonce":3100320732},{"type":"v1","addr":"172.21.15.139:6821","nonce":3100320732}]},"public_addr":"172.21.15.139:6817/3100320732","cluster_addr":"172.21.15.139:6819/3100320732","heartbeat_back_addr":"172.21.15.139:6823/3100320732","heartbeat_front_addr":"172.21.15.139:6821/3100320732","state":["exists","up"]},{"osd":9,"uuid":"ffaf81c6-340a-416a-8c8d-ed179d853403","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6827","nonce":2687652151}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6829","nonce":2687652151}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6833","nonce":2687652151}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2687652151},{"type":"v1","addr":"172.21.15.96:6831","nonce":2687652151}]},"public_addr":"172.21.15.96:6827/2687652151","cluster_addr":"172.21.15.96:6829/2687652151","heartbeat_back_addr":"172.21.15.96:6833/2687652151","heartbeat_front_addr":"172.21.15.96:6831/2687652151","state":["exists","up"]},{"osd":10,"uuid":"872e6405-0b55-4088-ab33-a8a3f4b7ac3e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6825","nonce":3642406379}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6827","nonce":3642406379}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6831","nonce":3642406379}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":3642406379},{"type":"v1","addr":"172.21.15.97:6829","nonce":3642406379}]},"public_addr":"172.21.15.97:6825/3642406379","cluster_addr":"172.21.15.97:6827/3642406379","heartbeat_back_addr":"172.21.15.97:6831/3642406379","heartbeat_front_addr":"172.21.15.97:6829/3642406379","state":["exists","up"]},{"osd":11,"uuid":"7a27fcf8-65a4-491e-9f62-8e597415e90c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":83,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6824","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6825","nonce":1534901897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6826","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6827","nonce":1534901897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6830","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6831","nonce":1534901897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.139:6828","nonce":1534901897},{"type":"v1","addr":"172.21.15.139:6829","nonce":1534901897}]},"public_addr":"172.21.15.139:6825/1534901897","cluster_addr":"172.21.15.139:6827/1534901897","heartbeat_back_addr":"172.21.15.139:6831/1534901897","heartbeat_front_addr":"172.21.15.139:6829/1534901897","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:48:44.321667+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:09.139950+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:33.822520+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:49:58.660962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:23.185979+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:50:48.023395+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:13.427771+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:51:38.382186+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:03.831828+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:29.437939+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:52:54.564243+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T20:53:20.431850+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.33","mappings":[{"from":5,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.96:0/2385969372":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/1528469114":"2024-04-26T20:57:38.516139+0000","172.21.15.96:6800/23858312":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/995634760":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/1163862190":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2305167259":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6801/3915899740":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/1496220452":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/111790105":"2024-04-26T20:46:41.969698+0000","172.21.15.96:0/2748921341":"2024-04-26T20:46:41.969698+0000","172.21.15.96:6801/23858312":"2024-04-26T20:57:38.516139+0000","172.21.15.96:0/2934738303":"2024-04-26T20:57:38.516139+0000","172.21.15.96:6800/1894355014":"2024-04-26T20:47:09.141019+0000","172.21.15.96:0/2265596716":"2024-04-26T20:47:09.141019+0000","172.21.15.96:6800/3915899740":"2024-04-26T20:46:41.969698+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":16,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T21:08:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f83240a3db0 msgr2=0x7f83240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f83240a3db0 0x7f83240a6270 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8334006010 tx=0x7f833401a040 comp rx=0 tx=0).stop 2024-04-25T21:08:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 msgr2=0x7f833819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:31.213 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f833819da10 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f833000b970 tx=0x7f833000bd30 comp rx=0 tx=0).stop 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 shutdown_connections 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f83240a3db0 0x7f83240a6270 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8338108a30 0x7f833819da10 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f833806a750 0x7f8338199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 --2- 172.21.15.96:0/2225722945 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f833806b0f0 0x7f8338199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 >> 172.21.15.96:0/2225722945 conn(0x7f83380788f0 msgr2=0x7f83380ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 shutdown_connections 2024-04-25T21:08:31.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:31.214+0000 7f8340188700 1 -- 172.21.15.96:0/2225722945 wait complete. 2024-04-25T21:08:31.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:31 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/2225722945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:31.613 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:31 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/2225722945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:31.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:31 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/2225722945' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-25T21:08:31.876 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-25T21:08:32.117 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:32.492 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:32 smithi096 ceph-mon[29288]: pgmap v338: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:08:32.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:32 smithi097 ceph-mon[32667]: pgmap v338: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:08:32.622 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:32 smithi139 ceph-mon[33027]: pgmap v338: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 767 B/s wr, 8 op/s 2024-04-25T21:08:33.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 -- 172.21.15.96:0/191245775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101dc0 msgr2=0x7f88bc10f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 --2- 172.21.15.96:0/191245775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101dc0 0x7f88bc10f830 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f88b400b0a0 tx=0x7f88b400b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 -- 172.21.15.96:0/191245775 shutdown_connections 2024-04-25T21:08:33.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 --2- 172.21.15.96:0/191245775 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101dc0 0x7f88bc10f830 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.721 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 --2- 172.21.15.96:0/191245775 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc107b20 0x7f88bc107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 --2- 172.21.15.96:0/191245775 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc101400 0x7f88bc101880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 -- 172.21.15.96:0/191245775 >> 172.21.15.96:0/191245775 conn(0x7f88bc076c60 msgr2=0x7f88bc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 -- 172.21.15.96:0/191245775 shutdown_connections 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.715+0000 7f88c11a6700 1 -- 172.21.15.96:0/191245775 wait complete. 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.716+0000 7f88c11a6700 1 Processor -- start 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.716+0000 7f88c11a6700 1 -- start start 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 0x7f88bc199100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc101dc0 0x7f88bc199640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc107b20 0x7f88bc19d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc112cb0 con 0x7f88bc101400 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc112b30 con 0x7f88bc107b20 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f88bc112e30 con 0x7f88bc101dc0 2024-04-25T21:08:33.722 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 0x7f88bc199100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 0x7f88bc199100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:47474/0 (socket says 172.21.15.96:47474) 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 -- 172.21.15.96:0/3466993479 learned_addr learned my addr 172.21.15.96:0/3466993479 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88ba59c700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc101dc0 0x7f88bc199640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bb59e700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc107b20 0x7f88bc19d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 -- 172.21.15.96:0/3466993479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc101dc0 msgr2=0x7f88bc199640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc101dc0 0x7f88bc199640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 -- 172.21.15.96:0/3466993479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc107b20 msgr2=0x7f88bc19d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc107b20 0x7f88bc19d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 -- 172.21.15.96:0/3466993479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88bc19e0c0 con 0x7f88bc101400 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88bad9d700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 0x7f88bc199100 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b500 tx=0x7f88ac00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac015020 con 0x7f88bc101400 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88bc19e3b0 con 0x7f88bc101400 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88ac011ce0 con 0x7f88bc101400 2024-04-25T21:08:33.723 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.717+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac010c80 con 0x7f88bc101400 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.718+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88bc1a5cb0 con 0x7f88bc101400 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.718+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88bc069730 con 0x7f88bc101400 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.719+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f88ac018020 con 0x7f88bc101400 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.720+0000 7f88a3fff700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f88a40a3db0 0x7f88a40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.720+0000 7f88ba59c700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f88a40a3db0 0x7f88a40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.720+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8455+0+0 (secure 0 0 0) 0x7f88ac0d0380 con 0x7f88bc101400 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.721+0000 7f88ba59c700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f88a40a3db0 0x7f88a40a6270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88bc077180 tx=0x7f88b000b410 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:33.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.724+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88ac01f020 con 0x7f88bc101400 2024-04-25T21:08:33.965 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.965+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f88bc19a410 con 0x7f88bc101400 2024-04-25T21:08:33.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.967+0000 7f88a3fff700 1 -- 172.21.15.96:0/3466993479 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v36) ==== 161+0+0 (secure 0 0 0) 0x7f88ac09cb70 con 0x7f88bc101400 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f88a40a3db0 msgr2=0x7f88a40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f88a40a3db0 0x7f88a40a6270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f88bc077180 tx=0x7f88b000b410 comp rx=0 tx=0).stop 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 msgr2=0x7f88bc199100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 0x7f88bc199100 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00b500 tx=0x7f88ac00b810 comp rx=0 tx=0).stop 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 shutdown_connections 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f88a40a3db0 0x7f88a40a6270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f88bc107b20 0x7f88bc19d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88bc101400 0x7f88bc199100 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 --2- 172.21.15.96:0/3466993479 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f88bc101dc0 0x7f88bc199640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 >> 172.21.15.96:0/3466993479 conn(0x7f88bc076c60 msgr2=0x7f88bc0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 shutdown_connections 2024-04-25T21:08:33.971 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:33.970+0000 7f88c11a6700 1 -- 172.21.15.96:0/3466993479 wait complete. 2024-04-25T21:08:33.972 INFO:teuthology.orchestra.run.smithi096.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T21:08:34.245 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a[29284]: 2024-04-25T21:08:33.966+0000 7f90f32bf700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:08:34.363 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e[62282]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:34.363 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e[62282]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:34.363 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mds[62287]: -- 172.21.15.97:0/2874372999 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55aa884f7a40 con 0x55aa884cb000 2024-04-25T21:08:34.363 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e[62282]: starting mds.e at 2024-04-25T21:08:34.363 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:34.363 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mds[62912]: -- 172.21.15.97:0/3006036849 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bac04bda40 con 0x55bac0491000 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: starting mds.b at 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h[62600]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h[62600]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mds[62604]: -- 172.21.15.97:0/3822803649 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557097965a40 con 0x557097939000 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h[62600]: starting mds.h at 2024-04-25T21:08:34.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: pgmap v339: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:08:34.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3466993479' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:08:34.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:08:34.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:08:34.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3466993479' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:08:34.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T21:08:34.365 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:34 smithi097 ceph-mon[32667]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T21:08:34.372 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i[63600]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:34.372 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i[63600]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:34.372 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mds[63604]: -- 172.21.15.139:0/587221538 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a4ca517a40 con 0x55a4ca4ea800 2024-04-25T21:08:34.372 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i[63600]: starting mds.i at 2024-04-25T21:08:34.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: pgmap v339: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:08:34.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3466993479' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3466993479' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mon[33027]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mds[63297]: -- 172.21.15.139:0/610702474 <== mon.1 v2:172.21.15.139:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b50d1da40 con 0x560b50cf0c00 2024-04-25T21:08:34.373 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: starting mds.c at 2024-04-25T21:08:34.374 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f[62986]: ignoring --setuser ceph since I am not root 2024-04-25T21:08:34.374 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f[62986]: ignoring --setgroup ceph since I am not root 2024-04-25T21:08:34.374 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-mds[62990]: -- 172.21.15.139:0/1924995776 <== mon.0 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5629cbcd1a40 con 0x5629cbca4800 2024-04-25T21:08:34.374 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:08:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f[62986]: starting mds.f at 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: pgmap v339: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 767 B/s wr, 6 op/s 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3466993479' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3466993479' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: osdmap e104: 12 total, 12 up, 12 in 2024-04-25T21:08:34.499 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:34 smithi096 ceph-mon[29288]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-04-25T21:08:34.619 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T21:08:34.859 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: pgmap v341: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6834/3981536249,v1:172.21.15.139:6835/3981536249] up:boot 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6834/2759850455,v1:172.21.15.97:6835/2759850455] up:boot 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6836/1856069538,v1:172.21.15.139:6837/1856069538] up:boot 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6836/1565218512,v1:172.21.15.97:6838/1565218512] up:boot 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.97:6837/1195963447,v1:172.21.15.97:6839/1195963447] up:boot 2024-04-25T21:08:36.224 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: mds.? [v2:172.21.15.139:6838/460588393,v1:172.21.15.139:6839/460588393] up:boot 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:08:36.225 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:35 smithi096 ceph-mon[29288]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: pgmap v341: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6834/3981536249,v1:172.21.15.139:6835/3981536249] up:boot 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6834/2759850455,v1:172.21.15.97:6835/2759850455] up:boot 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6836/1856069538,v1:172.21.15.139:6837/1856069538] up:boot 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6836/1565218512,v1:172.21.15.97:6838/1565218512] up:boot 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.97:6837/1195963447,v1:172.21.15.97:6839/1195963447] up:boot 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: mds.? [v2:172.21.15.139:6838/460588393,v1:172.21.15.139:6839/460588393] up:boot 2024-04-25T21:08:36.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T21:08:36.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:08:36.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:08:36.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:08:36.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:08:36.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:08:36.364 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:35 smithi097 ceph-mon[32667]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: pgmap v341: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6834/3981536249,v1:172.21.15.139:6835/3981536249] up:boot 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6834/2759850455,v1:172.21.15.97:6835/2759850455] up:boot 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6836/1856069538,v1:172.21.15.139:6837/1856069538] up:boot 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6836/1565218512,v1:172.21.15.97:6838/1565218512] up:boot 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.97:6837/1195963447,v1:172.21.15.97:6839/1195963447] up:boot 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: mds.? [v2:172.21.15.139:6838/460588393,v1:172.21.15.139:6839/460588393] up:boot 2024-04-25T21:08:36.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-04-25T21:08:36.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-25T21:08:36.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-25T21:08:36.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-25T21:08:36.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-25T21:08:36.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-25T21:08:36.373 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:35 smithi139 ceph-mon[33027]: from='mgr.15147 172.21.15.96:0/3056255577' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 -- 172.21.15.96:0/4079885818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 msgr2=0x7f03d8105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 --2- 172.21.15.96:0/4079885818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d8105910 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03d000b0a0 tx=0x7f03d000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 -- 172.21.15.96:0/4079885818 shutdown_connections 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 --2- 172.21.15.96:0/4079885818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d8105910 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 --2- 172.21.15.96:0/4079885818 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03d8107970 0x7f03d806b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 --2- 172.21.15.96:0/4079885818 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 0x7f03d806bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.611 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 -- 172.21.15.96:0/4079885818 >> 172.21.15.96:0/4079885818 conn(0x7f03d80788f0 msgr2=0x7f03d8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 -- 172.21.15.96:0/4079885818 shutdown_connections 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 -- 172.21.15.96:0/4079885818 wait complete. 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.606+0000 7f03e0248700 1 Processor -- start 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 -- start start 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 0x7f03d819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d819fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03d8107970 0x7f03d81a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03d81191e0 con 0x7f03d8105480 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8119060 con 0x7f03d806b740 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f03d8119360 con 0x7f03d8107970 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d819fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d819fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36150/0 (socket says 172.21.15.96:36150) 2024-04-25T21:08:36.612 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 -- 172.21.15.96:0/3735761353 learned_addr learned my addr 172.21.15.96:0/3735761353 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03de7e5700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03d8107970 0x7f03d81a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03ddfe4700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 0x7f03d819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 -- 172.21.15.96:0/3735761353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03d8107970 msgr2=0x7f03d81a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03d8107970 0x7f03d81a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 -- 172.21.15.96:0/3735761353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 msgr2=0x7f03d819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 0x7f03d819f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 -- 172.21.15.96:0/3735761353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d81a4620 con 0x7f03d8105480 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03ddfe4700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 0x7f03d819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03dd7e3700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d819fb50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c9b0 tx=0x7f03c800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8007b00 con 0x7f03d8105480 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c8007ca0 con 0x7f03d8105480 2024-04-25T21:08:36.613 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d81a4910 con 0x7f03d8105480 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.607+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c8018730 con 0x7f03d8105480 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.608+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d81ac190 con 0x7f03d8105480 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.608+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d8069730 con 0x7f03d8105480 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.610+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f03c800f450 con 0x7f03d8105480 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.610+0000 7f03ceffd700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f03c40a3db0 0x7f03c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.610+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8669+0+0 (secure 0 0 0) 0x7f03c80d0c50 con 0x7f03d8105480 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.610+0000 7f03ddfe4700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f03c40a3db0 0x7f03c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.611+0000 7f03ddfe4700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f03c40a3db0 0x7f03c40a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009b50 tx=0x7f03d4009550 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:36.614 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.614+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f03c8014030 con 0x7f03d8105480 2024-04-25T21:08:36.857 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.856+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f03d81ac450 con 0x7f03d8105480 2024-04-25T21:08:36.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.980+0000 7f03ceffd700 1 -- 172.21.15.96:0/3735761353 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) ==== 104+0+0 (secure 0 0 0) 0x7f03c809d2f0 con 0x7f03d8105480 2024-04-25T21:08:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f03c40a3db0 msgr2=0x7f03c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f03c40a3db0 0x7f03c40a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f03d4009b50 tx=0x7f03d4009550 comp rx=0 tx=0).stop 2024-04-25T21:08:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 msgr2=0x7f03d819fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d819fb50 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03c800c9b0 tx=0x7f03c800cd70 comp rx=0 tx=0).stop 2024-04-25T21:08:36.983 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 shutdown_connections 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f03c40a3db0 0x7f03c40a6270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f03d8107970 0x7f03d81a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f03d806b740 0x7f03d819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 --2- 172.21.15.96:0/3735761353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f03d8105480 0x7f03d819fb50 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 >> 172.21.15.96:0/3735761353 conn(0x7f03d80788f0 msgr2=0x7f03d8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 shutdown_connections 2024-04-25T21:08:36.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:36.983+0000 7f03e0248700 1 -- 172.21.15.96:0/3735761353 wait complete. 2024-04-25T21:08:37.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:36 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3735761353' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:08:37.362 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:36 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3735761353' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:08:37.371 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:36 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3735761353' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-25T21:08:37.634 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T21:08:37.873 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:38.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:37 smithi096 ceph-mon[29288]: pgmap v342: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:38.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:37 smithi096 ceph-mon[29288]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:08:38.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:37 smithi096 ceph-mon[29288]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:08:38.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:37 smithi096 ceph-mon[29288]: Cluster is now healthy 2024-04-25T21:08:38.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:37 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/3735761353' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:08:38.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:37 smithi096 ceph-mon[29288]: fsmap 12 up:standby 2024-04-25T21:08:38.362 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:37 smithi097 ceph-mon[32667]: pgmap v342: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:38.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:37 smithi097 ceph-mon[32667]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:08:38.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:37 smithi097 ceph-mon[32667]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:08:38.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:37 smithi097 ceph-mon[32667]: Cluster is now healthy 2024-04-25T21:08:38.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:37 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/3735761353' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:08:38.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:37 smithi097 ceph-mon[32667]: fsmap 12 up:standby 2024-04-25T21:08:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:37 smithi139 ceph-mon[33027]: pgmap v342: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:37 smithi139 ceph-mon[33027]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-25T21:08:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:37 smithi139 ceph-mon[33027]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-25T21:08:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:37 smithi139 ceph-mon[33027]: Cluster is now healthy 2024-04-25T21:08:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:37 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/3735761353' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-25T21:08:38.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:37 smithi139 ceph-mon[33027]: fsmap 12 up:standby 2024-04-25T21:08:39.487 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 -- 172.21.15.96:0/2071253831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8103450 msgr2=0x7f13b810f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 --2- 172.21.15.96:0/2071253831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8103450 0x7f13b810f8b0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f13b000b0a0 tx=0x7f13b000b3b0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 -- 172.21.15.96:0/2071253831 shutdown_connections 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 --2- 172.21.15.96:0/2071253831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8103450 0x7f13b810f8b0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 --2- 172.21.15.96:0/2071253831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8108a50 0x7f13b8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 --2- 172.21.15.96:0/2071253831 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8102a90 0x7f13b8102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 -- 172.21.15.96:0/2071253831 >> 172.21.15.96:0/2071253831 conn(0x7f13b8079ba0 msgr2=0x7f13b8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 -- 172.21.15.96:0/2071253831 shutdown_connections 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 -- 172.21.15.96:0/2071253831 wait complete. 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 Processor -- start 2024-04-25T21:08:39.488 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.487+0000 7f13be50a700 1 -- start start 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13be50a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8102a90 0x7f13b8199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13be50a700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8103450 0x7f13b81995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13be50a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8108a50 0x7f13b819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13be50a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8112c20 con 0x7f13b8108a50 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13be50a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8112aa0 con 0x7f13b8102a90 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13be50a700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f13b8112da0 con 0x7f13b8103450 2024-04-25T21:08:39.490 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8102a90 0x7f13b8199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8108a50 0x7f13b819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13b77fe700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8103450 0x7f13b81995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13b7fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8102a90 0x7f13b8199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.96:41610/0 (socket says 172.21.15.96:41610) 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13b7fff700 1 -- 172.21.15.96:0/1700530807 learned_addr learned my addr 172.21.15.96:0/1700530807 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 -- 172.21.15.96:0/1700530807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8103450 msgr2=0x7f13b81995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8103450 0x7f13b81995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 -- 172.21.15.96:0/1700530807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8102a90 msgr2=0x7f13b8199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8102a90 0x7f13b8199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 -- 172.21.15.96:0/1700530807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13b819e0a0 con 0x7f13b8108a50 2024-04-25T21:08:39.491 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13bcaa7700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8108a50 0x7f13b819d960 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f13b000b070 tx=0x7f13b0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.488+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0021070 con 0x7f13b8108a50 2024-04-25T21:08:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.489+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13b819e330 con 0x7f13b8108a50 2024-04-25T21:08:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.489+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b0007b80 con 0x7f13b8108a50 2024-04-25T21:08:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.489+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b0005040 con 0x7f13b8108a50 2024-04-25T21:08:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.489+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13b81a5c10 con 0x7f13b8108a50 2024-04-25T21:08:39.492 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.490+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13b8069730 con 0x7f13b8108a50 2024-04-25T21:08:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.491+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13b0024070 con 0x7f13b8108a50 2024-04-25T21:08:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.491+0000 7f13b57fa700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f13a00a3db0 0x7f13a00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.492+0000 7f13b7fff700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f13a00a3db0 0x7f13a00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.492+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8669+0+0 (secure 0 0 0) 0x7f13b000f020 con 0x7f13b8108a50 2024-04-25T21:08:39.493 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.492+0000 7f13b7fff700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f13a00a3db0 0x7f13a00a6270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13a8005950 tx=0x7f13a800b500 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:39.495 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.495+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13b002b020 con 0x7f13b8108a50 2024-04-25T21:08:39.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.693+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f13b802df80 con 0x7f13b8108a50 2024-04-25T21:08:39.976 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.976+0000 7f13b57fa700 1 -- 172.21.15.96:0/1700530807 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v105) ==== 181+0+0 (secure 0 0 0) 0x7f13b009e860 con 0x7f13b8108a50 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f13a00a3db0 msgr2=0x7f13a00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f13a00a3db0 0x7f13a00a6270 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13a8005950 tx=0x7f13a800b500 comp rx=0 tx=0).stop 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8108a50 msgr2=0x7f13b819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8108a50 0x7f13b819d960 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f13b000b070 tx=0x7f13b0004ce0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 shutdown_connections 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f13a00a3db0 0x7f13a00a6270 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13b8108a50 0x7f13b819d960 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.980 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f13b8102a90 0x7f13b8199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.979+0000 7f13be50a700 1 --2- 172.21.15.96:0/1700530807 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f13b8103450 0x7f13b81995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.980+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 >> 172.21.15.96:0/1700530807 conn(0x7f13b8079ba0 msgr2=0x7f13b81017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.980+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 shutdown_connections 2024-04-25T21:08:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:39.980+0000 7f13be50a700 1 -- 172.21.15.96:0/1700530807 wait complete. 2024-04-25T21:08:39.981 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_metadata' removed 2024-04-25T21:08:40.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:39 smithi096 ceph-mon[29288]: pgmap v343: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:40.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:39 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1700530807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:08:40.362 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:39 smithi097 ceph-mon[32667]: pgmap v343: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:40.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:39 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1700530807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:08:40.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:39 smithi139 ceph-mon[33027]: pgmap v343: 97 pgs: 97 active+clean; 67 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-04-25T21:08:40.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:39 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1700530807' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:08:40.714 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f shell --fsid b713778a-0344-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T21:08:40.953 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/mon.a/config 2024-04-25T21:08:41.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:40 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/1700530807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:08:41.249 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:40 smithi096 ceph-mon[29288]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T21:08:41.362 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:40 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/1700530807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:08:41.363 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:40 smithi097 ceph-mon[32667]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T21:08:41.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:40 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/1700530807' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:08:41.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:40 smithi139 ceph-mon[33027]: osdmap e105: 12 total, 12 up, 12 in 2024-04-25T21:08:42.248 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:41 smithi096 ceph-mon[29288]: pgmap v345: 65 pgs: 65 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:42.362 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:41 smithi097 ceph-mon[32667]: pgmap v345: 65 pgs: 65 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:42.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:41 smithi139 ceph-mon[33027]: pgmap v345: 65 pgs: 65 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:42.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 -- 172.21.15.96:0/1555615084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc108a50 msgr2=0x7f96cc108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.761 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 --2- 172.21.15.96:0/1555615084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc108a50 0x7f96cc108e30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009990 tx=0x7f96bc009ca0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 -- 172.21.15.96:0/1555615084 shutdown_connections 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 --2- 172.21.15.96:0/1555615084 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96cc103450 0x7f96cc10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 --2- 172.21.15.96:0/1555615084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc108a50 0x7f96cc108e30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 --2- 172.21.15.96:0/1555615084 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc102a90 0x7f96cc102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 -- 172.21.15.96:0/1555615084 >> 172.21.15.96:0/1555615084 conn(0x7f96cc079ba0 msgr2=0x7f96cc079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 -- 172.21.15.96:0/1555615084 shutdown_connections 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 -- 172.21.15.96:0/1555615084 wait complete. 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 Processor -- start 2024-04-25T21:08:42.762 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.761+0000 7f96d231c700 1 -- start start 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d231c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 0x7f96cc074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d231c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96cc103450 0x7f96cc075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d231c700 1 --2- >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc108a50 0x7f96cc0757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d231c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc112be0 con 0x7f96cc102a90 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d231c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc112a60 con 0x7f96cc103450 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d231c700 1 -- --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc112d60 con 0x7f96cc108a50 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 0x7f96cc074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.764 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 0x7f96cc074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:36194/0 (socket says 172.21.15.96:36194) 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 -- 172.21.15.96:0/908801864 learned_addr learned my addr 172.21.15.96:0/908801864 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96d08b9700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc108a50 0x7f96cc0757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cb7fe700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96cc103450 0x7f96cc075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 -- 172.21.15.96:0/908801864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc108a50 msgr2=0x7f96cc0757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc108a50 0x7f96cc0757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 -- 172.21.15.96:0/908801864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96cc103450 msgr2=0x7f96cc075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96cc103450 0x7f96cc075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 -- 172.21.15.96:0/908801864 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1a75e0 con 0x7f96cc102a90 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.762+0000 7f96cbfff700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 0x7f96cc074d40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0057c0 tx=0x7f96bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.763+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc027070 con 0x7f96cc102a90 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.763+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1a7870 con 0x7f96cc102a90 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.763+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96bc003f20 con 0x7f96cc102a90 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.763+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96bc022b80 con 0x7f96cc102a90 2024-04-25T21:08:42.765 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.763+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1a7da0 con 0x7f96cc102a90 2024-04-25T21:08:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.764+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96cc069730 con 0x7f96cc102a90 2024-04-25T21:08:42.766 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.765+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f96bc02a070 con 0x7f96cc102a90 2024-04-25T21:08:42.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.765+0000 7f96c97fa700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f96b40a3db0 0x7f96b40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T21:08:42.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.765+0000 7f96cb7fe700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f96b40a3db0 0x7f96b40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T21:08:42.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.766+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8173+0+0 (secure 0 0 0) 0x7f96bc0d1d50 con 0x7f96cc102a90 2024-04-25T21:08:42.767 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.766+0000 7f96cb7fe700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f96b40a3db0 0x7f96b40a6270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96cc07a1a0 tx=0x7f96c000a380 comp rx=0 tx=0).ready entity=mgr.15147 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T21:08:42.769 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.769+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f96bc031020 con 0x7f96cc102a90 2024-04-25T21:08:42.967 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.966+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f96cc02df80 con 0x7f96cc102a90 2024-04-25T21:08:42.997 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:42.997+0000 7f96c97fa700 1 -- 172.21.15.96:0/908801864 <== mon.0 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v106) ==== 169+0+0 (secure 0 0 0) 0x7f96bc013070 con 0x7f96cc102a90 2024-04-25T21:08:43.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f96b40a3db0 msgr2=0x7f96b40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f96b40a3db0 0x7f96b40a6270 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96cc07a1a0 tx=0x7f96c000a380 comp rx=0 tx=0).stop 2024-04-25T21:08:43.001 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 msgr2=0x7f96cc074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 0x7f96cc074d40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f96bc0057c0 tx=0x7f96bc0057f0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 shutdown_connections 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:6800/3924141367,v1:172.21.15.96:6801/3924141367] conn(0x7f96b40a3db0 0x7f96b40a6270 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f96cc07a1a0 tx=0x7f96c000a380 comp rx=0 tx=0).stop 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] conn(0x7f96cc108a50 0x7f96cc0757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96cc102a90 0x7f96cc074d40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 --2- 172.21.15.96:0/908801864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f96cc103450 0x7f96cc075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T21:08:43.002 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.001+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 >> 172.21.15.96:0/908801864 conn(0x7f96cc079ba0 msgr2=0x7f96cc1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T21:08:43.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.002+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 shutdown_connections 2024-04-25T21:08:43.003 INFO:teuthology.orchestra.run.smithi096.stderr:2024-04-25T21:08:43.002+0000 7f96d231c700 1 -- 172.21.15.96:0/908801864 wait complete. 2024-04-25T21:08:43.004 INFO:teuthology.orchestra.run.smithi096.stderr:pool 'cephfs_data' removed 2024-04-25T21:08:43.019 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:42 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/908801864' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:08:43.362 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:42 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/908801864' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:08:43.371 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:42 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/908801864' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-25T21:08:43.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:43.764 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:43.794 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:08:43.823 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-25T21:08:43.823 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:08:43.839 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:08:43.854 DEBUG:teuthology.orchestra.run.smithi139:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-25T21:08:43.878 INFO:tasks.cephadm:Stopping all daemons... 2024-04-25T21:08:43.878 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-25T21:08:43.878 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a 2024-04-25T21:08:43.999 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:43 smithi096 systemd[1]: Stopping Ceph mon.a for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:44.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:44 smithi097 ceph-mon[32667]: pgmap v346: 65 pgs: 65 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:44.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:44 smithi097 ceph-mon[32667]: from='client.? 172.21.15.96:0/908801864' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:08:44.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:44 smithi097 ceph-mon[32667]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T21:08:44.121 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:44 smithi139 ceph-mon[33027]: pgmap v346: 65 pgs: 65 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:44.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:44 smithi139 ceph-mon[33027]: from='client.? 172.21.15.96:0/908801864' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:08:44.122 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:44 smithi139 ceph-mon[33027]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T21:08:44.470 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 ceph-mon[29288]: pgmap v346: 65 pgs: 65 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail 2024-04-25T21:08:44.471 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 ceph-mon[29288]: from='client.? 172.21.15.96:0/908801864' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-25T21:08:44.471 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 ceph-mon[29288]: osdmap e106: 12 total, 12 up, 12 in 2024-04-25T21:08:44.471 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a[29284]: 2024-04-25T21:08:44.205+0000 7f90f8aca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:08:44.471 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a[29284]: 2024-04-25T21:08:44.205+0000 7f90f8aca700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-25T21:08:44.749 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 podman[115732]: 2024-04-25 21:08:44.471565891 +0000 UTC m=+0.399234144 container died 45ed8be0b8dbe0689e387f3bd278cc55f09bdc23d366e67456935626f479001d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-25T21:08:45.107 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 podman[115732]: 2024-04-25 21:08:44.864706131 +0000 UTC m=+0.792374384 container cleanup 45ed8be0b8dbe0689e387f3bd278cc55f09bdc23d366e67456935626f479001d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T21:08:45.107 INFO:journalctl@ceph.mon.a.smithi096.stdout:Apr 25 21:08:44 smithi096 bash[115732]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-a 2024-04-25T21:08:45.247 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.a.service' 2024-04-25T21:08:45.304 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:45.304 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-25T21:08:45.304 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-25T21:08:45.304 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.b 2024-04-25T21:08:45.612 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:45 smithi097 systemd[1]: Stopping Ceph mon.b for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:46.112 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-b[32662]: 2024-04-25T21:08:45.685+0000 7ff0a5da3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:08:46.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-b[32662]: 2024-04-25T21:08:45.685+0000 7ff0a5da3700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-25T21:08:46.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:45 smithi097 podman[69076]: 2024-04-25 21:08:45.911112296 +0000 UTC m=+0.410208981 container died 6bdc7af8f19230b01a7db11592d8abdc1cce468e8121215bdf87c0bc51726230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-b, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T21:08:46.554 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:46 smithi097 podman[69076]: 2024-04-25 21:08:46.294729405 +0000 UTC m=+0.793826098 container cleanup 6bdc7af8f19230b01a7db11592d8abdc1cce468e8121215bdf87c0bc51726230 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=) 2024-04-25T21:08:46.555 INFO:journalctl@ceph.mon.b.smithi097.stdout:Apr 25 21:08:46 smithi097 bash[69076]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-b 2024-04-25T21:08:46.719 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.b.service' 2024-04-25T21:08:46.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:46.765 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-25T21:08:46.765 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-25T21:08:46.765 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.c 2024-04-25T21:08:47.115 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:46 smithi139 systemd[1]: Stopping Ceph mon.c for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:47.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c[33023]: 2024-04-25T21:08:47.114+0000 7fbce264d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:08:47.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c[33023]: 2024-04-25T21:08:47.114+0000 7fbce264d700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-25T21:08:47.372 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:47 smithi139 podman[69929]: 2024-04-25 21:08:47.289392054 +0000 UTC m=+0.332054806 container died 7941e855b680ac30167dad977b9dfa4796b05b82644206b9b661abba9784f082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-04-25T21:08:47.841 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:47 smithi139 podman[69929]: 2024-04-25 21:08:47.673656593 +0000 UTC m=+0.716319333 container cleanup 7941e855b680ac30167dad977b9dfa4796b05b82644206b9b661abba9784f082 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:08:47.841 INFO:journalctl@ceph.mon.c.smithi139.stdout:Apr 25 21:08:47 smithi139 bash[69929]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mon-c 2024-04-25T21:08:48.047 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mon.c.service' 2024-04-25T21:08:48.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:48.096 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-25T21:08:48.096 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-25T21:08:48.096 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x 2024-04-25T21:08:48.410 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 21:08:48 smithi096 systemd[1]: Stopping Ceph mgr.x for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:48.702 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 21:08:48 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T21:08:48.409+0000 7f85c8544700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:08:48.702 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 21:08:48 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x[29518]: 2024-04-25T21:08:48.409+0000 7f85c8544700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-04-25T21:08:48.702 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 21:08:48 smithi096 podman[115856]: 2024-04-25 21:08:48.441392244 +0000 UTC m=+0.153155660 container died 8c60889ad849595d42378f23a3798360f1dddb13916c86a305d8eeeb96660985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-25T21:08:48.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 21:08:48 smithi096 podman[115856]: 2024-04-25 21:08:48.810967341 +0000 UTC m=+0.522730755 container cleanup 8c60889ad849595d42378f23a3798360f1dddb13916c86a305d8eeeb96660985 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:d69f13d827952bb07bb9b7973bca14f97d764d6f, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:08:48.999 INFO:journalctl@ceph.mgr.x.smithi096.stdout:Apr 25 21:08:48 smithi096 bash[115856]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-x 2024-04-25T21:08:49.234 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-25T21:08:49.268 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:49.269 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-25T21:08:49.269 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-25T21:08:49.269 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.y 2024-04-25T21:08:49.595 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 21:08:49 smithi097 systemd[1]: Stopping Ceph mgr.y for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:49.862 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 21:08:49 smithi097 podman[69197]: 2024-04-25 21:08:49.595787816 +0000 UTC m=+0.124671575 container died cb310225ad670517c5b1aab87c835d4fc6523c6601b88388d1c661b6b065a44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T21:08:50.261 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 21:08:49 smithi097 podman[69197]: 2024-04-25 21:08:49.952862399 +0000 UTC m=+0.481746209 container cleanup cb310225ad670517c5b1aab87c835d4fc6523c6601b88388d1c661b6b065a44e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:08:50.262 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Apr 25 21:08:49 smithi097 bash[69197]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-y 2024-04-25T21:08:50.424 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-25T21:08:50.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:50.457 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-25T21:08:50.457 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-25T21:08:50.458 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.z 2024-04-25T21:08:50.781 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 21:08:50 smithi139 systemd[1]: Stopping Ceph mgr.z for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:51.087 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 21:08:50 smithi139 podman[70049]: 2024-04-25 21:08:50.78219873 +0000 UTC m=+0.127753565 container died e25241a14a5db2ad206f970a8208fd67b105e8bc990d31ea2610fbbe31dad2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:08:51.379 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 21:08:51 smithi139 podman[70049]: 2024-04-25 21:08:51.169961749 +0000 UTC m=+0.515516584 container cleanup e25241a14a5db2ad206f970a8208fd67b105e8bc990d31ea2610fbbe31dad2a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-25T21:08:51.379 INFO:journalctl@ceph.mgr.z.smithi139.stdout:Apr 25 21:08:51 smithi139 bash[70049]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mgr-z 2024-04-25T21:08:51.636 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-25T21:08:51.670 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:51.670 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-25T21:08:51.671 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-25T21:08:51.671 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.0 2024-04-25T21:08:51.964 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:51 smithi096 systemd[1]: Stopping Ceph osd.0 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:08:52.248 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0[43881]: 2024-04-25T21:08:51.964+0000 7ff54d87a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:08:52.249 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0[43881]: 2024-04-25T21:08:51.964+0000 7ff54d87a700 -1 osd.0 106 *** Got signal Terminated *** 2024-04-25T21:08:52.249 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0[43881]: 2024-04-25T21:08:51.964+0000 7ff54d87a700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:08:57.640 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:57 smithi096 podman[115976]: 2024-04-25 21:08:57.342695152 +0000 UTC m=+5.482626352 container died 0383cc1c849ba1ad13d2f71e43710d1a1ab7a6f8c5b1f4be6370f8ca86b01b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-25T21:08:57.965 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:57 smithi096 podman[115976]: 2024-04-25 21:08:57.731524983 +0000 UTC m=+5.871456230 container cleanup 0383cc1c849ba1ad13d2f71e43710d1a1ab7a6f8c5b1f4be6370f8ca86b01b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:08:57.965 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:57 smithi096 bash[115976]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0 2024-04-25T21:08:58.249 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:57 smithi096 podman[115993]: 2024-04-25 21:08:57.966017223 +0000 UTC m=+0.619454162 container remove 0383cc1c849ba1ad13d2f71e43710d1a1ab7a6f8c5b1f4be6370f8ca86b01b3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T21:08:58.569 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:58 smithi096 podman[116080]: 2024-04-25 21:08:58.20143071 +0000 UTC m=+0.018257981 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:08:58.569 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:58 smithi096 podman[116080]: 2024-04-25 21:08:58.325211612 +0000 UTC m=+0.142038868 container create 87da122b1d395a54c5532c2fa2ce07015b5d26a1bf8f8ea2b4317e0b7844c374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:08:58.865 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:58 smithi096 podman[116080]: 2024-04-25 21:08:58.569968035 +0000 UTC m=+0.386795292 container init 87da122b1d395a54c5532c2fa2ce07015b5d26a1bf8f8ea2b4317e0b7844c374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-25T21:08:58.865 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:58 smithi096 podman[116080]: 2024-04-25 21:08:58.573803253 +0000 UTC m=+0.390630510 container start 87da122b1d395a54c5532c2fa2ce07015b5d26a1bf8f8ea2b4317e0b7844c374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T21:08:58.865 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:58 smithi096 podman[116080]: 2024-04-25 21:08:58.669198759 +0000 UTC m=+0.486026024 container attach 87da122b1d395a54c5532c2fa2ce07015b5d26a1bf8f8ea2b4317e0b7844c374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-25T21:08:59.248 INFO:journalctl@ceph.osd.0.smithi096.stdout:Apr 25 21:08:58 smithi096 podman[116080]: 2024-04-25 21:08:58.866574883 +0000 UTC m=+0.683402152 container died 87da122b1d395a54c5532c2fa2ce07015b5d26a1bf8f8ea2b4317e0b7844c374 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T21:08:59.666 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.0.service' 2024-04-25T21:08:59.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:08:59.706 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-25T21:08:59.706 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-25T21:08:59.706 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.1 2024-04-25T21:09:00.040 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:08:59 smithi097 systemd[1]: Stopping Ceph osd.1 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:00.363 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1[37248]: 2024-04-25T21:09:00.040+0000 7f54abcc7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:00.363 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1[37248]: 2024-04-25T21:09:00.040+0000 7f54abcc7700 -1 osd.1 106 *** Got signal Terminated *** 2024-04-25T21:09:00.363 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1[37248]: 2024-04-25T21:09:00.040+0000 7f54abcc7700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:05.715 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:05 smithi097 podman[69334]: 2024-04-25 21:09:05.451727938 +0000 UTC m=+5.542217363 container died b57053a83abd81d1fd232a5106ecbb104513cae678376450d242742d10bcaac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-25T21:09:06.024 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:05 smithi097 podman[69334]: 2024-04-25 21:09:05.806680108 +0000 UTC m=+5.897169530 container cleanup b57053a83abd81d1fd232a5106ecbb104513cae678376450d242742d10bcaac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-25T21:09:06.024 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:05 smithi097 bash[69334]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1 2024-04-25T21:09:06.333 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69354]: 2024-04-25 21:09:06.02473453 +0000 UTC m=+0.571839118 container remove b57053a83abd81d1fd232a5106ecbb104513cae678376450d242742d10bcaac4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T21:09:06.613 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69444]: 2024-04-25 21:09:06.334128624 +0000 UTC m=+0.113832765 container create 20403851f10494627be0dc4a55ca91d1d0554e180c27508cd6741a92ae0e5368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD) 2024-04-25T21:09:06.613 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69444]: 2024-04-25 21:09:06.238337814 +0000 UTC m=+0.018041968 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:06.613 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69444]: 2024-04-25 21:09:06.553581605 +0000 UTC m=+0.333285748 container init 20403851f10494627be0dc4a55ca91d1d0554e180c27508cd6741a92ae0e5368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-25T21:09:06.613 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69444]: 2024-04-25 21:09:06.557082139 +0000 UTC m=+0.336786282 container start 20403851f10494627be0dc4a55ca91d1d0554e180c27508cd6741a92ae0e5368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:09:07.112 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69444]: 2024-04-25 21:09:06.644547335 +0000 UTC m=+0.424251486 container attach 20403851f10494627be0dc4a55ca91d1d0554e180c27508cd6741a92ae0e5368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T21:09:07.113 INFO:journalctl@ceph.osd.1.smithi097.stdout:Apr 25 21:09:06 smithi097 podman[69444]: 2024-04-25 21:09:06.847937266 +0000 UTC m=+0.627641419 container died 20403851f10494627be0dc4a55ca91d1d0554e180c27508cd6741a92ae0e5368 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416) 2024-04-25T21:09:07.470 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.1.service' 2024-04-25T21:09:07.509 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:07.509 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-25T21:09:07.509 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-25T21:09:07.509 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.2 2024-04-25T21:09:07.815 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:07 smithi139 systemd[1]: Stopping Ceph osd.2 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:08.122 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2[38159]: 2024-04-25T21:09:07.814+0000 7f43a6fdd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:08.122 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2[38159]: 2024-04-25T21:09:07.814+0000 7f43a6fdd700 -1 osd.2 106 *** Got signal Terminated *** 2024-04-25T21:09:08.122 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2[38159]: 2024-04-25T21:09:07.814+0000 7f43a6fdd700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:13.607 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:13 smithi139 podman[70187]: 2024-04-25 21:09:13.322505948 +0000 UTC m=+5.623553854 container died de77454f9250a62b0889b1838a0c082de7aa7bd2abdcede397323fe527687015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:09:13.874 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:13 smithi139 podman[70187]: 2024-04-25 21:09:13.706161613 +0000 UTC m=+6.007209521 container cleanup de77454f9250a62b0889b1838a0c082de7aa7bd2abdcede397323fe527687015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-25T21:09:13.874 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:13 smithi139 bash[70187]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2 2024-04-25T21:09:14.302 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:13 smithi139 podman[70205]: 2024-04-25 21:09:13.991146287 +0000 UTC m=+0.666169837 container remove de77454f9250a62b0889b1838a0c082de7aa7bd2abdcede397323fe527687015 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-25T21:09:14.610 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:14 smithi139 podman[70299]: 2024-04-25 21:09:14.205521361 +0000 UTC m=+0.018338499 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:14.611 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:14 smithi139 podman[70299]: 2024-04-25 21:09:14.34192631 +0000 UTC m=+0.154743424 container create 0973e0a3ed0f79e65b4ce68f54ff0f820a746cf9928ec5846a1c030342a70529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T21:09:14.872 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:14 smithi139 podman[70299]: 2024-04-25 21:09:14.611135823 +0000 UTC m=+0.423952948 container init 0973e0a3ed0f79e65b4ce68f54ff0f820a746cf9928ec5846a1c030342a70529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0) 2024-04-25T21:09:14.872 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:14 smithi139 podman[70299]: 2024-04-25 21:09:14.614535767 +0000 UTC m=+0.427352884 container start 0973e0a3ed0f79e65b4ce68f54ff0f820a746cf9928ec5846a1c030342a70529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:09:14.872 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:14 smithi139 podman[70299]: 2024-04-25 21:09:14.71090812 +0000 UTC m=+0.523725240 container attach 0973e0a3ed0f79e65b4ce68f54ff0f820a746cf9928ec5846a1c030342a70529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-25T21:09:15.204 INFO:journalctl@ceph.osd.2.smithi139.stdout:Apr 25 21:09:14 smithi139 podman[70299]: 2024-04-25 21:09:14.913469654 +0000 UTC m=+0.726286776 container died 0973e0a3ed0f79e65b4ce68f54ff0f820a746cf9928ec5846a1c030342a70529 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-25T21:09:15.526 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.2.service' 2024-04-25T21:09:15.567 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:15.567 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-25T21:09:15.567 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-25T21:09:15.567 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.3 2024-04-25T21:09:15.899 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:15 smithi096 systemd[1]: Stopping Ceph osd.3 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:16.248 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:15 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T21:09:15.899+0000 7fa21d793700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:16.249 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:15 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T21:09:15.899+0000 7fa21d793700 -1 osd.3 106 *** Got signal Terminated *** 2024-04-25T21:09:16.249 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:15 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T21:09:15.899+0000 7fa21d793700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:18.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:18.348+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:19.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:19.337+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:19.748 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:19 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T21:09:19.484+0000 7fa219d87700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.514052+0000 front 2024-04-25T21:08:54.514028+0000 (oldest deadline 2024-04-25T21:09:18.614230+0000) 2024-04-25T21:09:20.122 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:19.859+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:20.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:20.308+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:20.999 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:20 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3[50406]: 2024-04-25T21:09:20.520+0000 7fa219d87700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.514052+0000 front 2024-04-25T21:08:54.514028+0000 (oldest deadline 2024-04-25T21:09:18.614230+0000) 2024-04-25T21:09:21.271 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:20.877+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:21.598 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:21 smithi096 podman[116201]: 2024-04-25 21:09:21.312700001 +0000 UTC m=+5.540575456 container died 16e7dc02856430059ad8e7d49fda015f81dbc477324a5f24fc3a0dce86a5fb13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:09:21.598 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:21 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:21.348+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:21.621 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:21.270+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:21.915 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:21 smithi096 podman[116201]: 2024-04-25 21:09:21.698473267 +0000 UTC m=+5.926348719 container cleanup 16e7dc02856430059ad8e7d49fda015f81dbc477324a5f24fc3a0dce86a5fb13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T21:09:21.915 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:21 smithi096 bash[116201]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3 2024-04-25T21:09:22.249 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:21 smithi096 podman[116216]: 2024-04-25 21:09:21.916433124 +0000 UTC m=+0.602521114 container remove 16e7dc02856430059ad8e7d49fda015f81dbc477324a5f24fc3a0dce86a5fb13 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:09:22.249 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:22.136+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:22.288 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:21.872+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:22.619 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:22 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:22.394+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:22.619 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:22 smithi096 podman[116308]: 2024-04-25 21:09:22.34238248 +0000 UTC m=+0.101535736 container create c107157b971110414174031be77370fd82cff417ecf61c0d562140d0867c1777 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:09:22.619 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:22 smithi096 podman[116308]: 2024-04-25 21:09:22.258713981 +0000 UTC m=+0.017867251 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:22.619 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:22 smithi096 podman[116308]: 2024-04-25 21:09:22.552985226 +0000 UTC m=+0.312138497 container init c107157b971110414174031be77370fd82cff417ecf61c0d562140d0867c1777 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True) 2024-04-25T21:09:22.619 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:22 smithi096 podman[116308]: 2024-04-25 21:09:22.555980884 +0000 UTC m=+0.315134140 container start c107157b971110414174031be77370fd82cff417ecf61c0d562140d0867c1777 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-04-25T21:09:22.619 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:22 smithi096 podman[116308]: 2024-04-25 21:09:22.619609421 +0000 UTC m=+0.378762678 container attach c107157b971110414174031be77370fd82cff417ecf61c0d562140d0867c1777 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T21:09:22.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:22.287+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:22.999 INFO:journalctl@ceph.osd.3.smithi096.stdout:Apr 25 21:09:22 smithi096 podman[116308]: 2024-04-25 21:09:22.851592317 +0000 UTC m=+0.610745574 container died c107157b971110414174031be77370fd82cff417ecf61c0d562140d0867c1777 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-25T21:09:23.309 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:22.897+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:23.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:22 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:22.994+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:23.430 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:23 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:23.152+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:23.430 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:23 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:23.366+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:23.448 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.3.service' 2024-04-25T21:09:23.485 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:23.486 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-25T21:09:23.486 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-25T21:09:23.486 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.4 2024-04-25T21:09:23.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:23.308+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:23.783 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:23 smithi097 systemd[1]: Stopping Ceph osd.4 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:24.112 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:24 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:24.020+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:24.113 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4[43773]: 2024-04-25T21:09:23.782+0000 7fdb37cc6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:24.113 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4[43773]: 2024-04-25T21:09:23.782+0000 7fdb37cc6700 -1 osd.4 106 *** Got signal Terminated *** 2024-04-25T21:09:24.113 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:23 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4[43773]: 2024-04-25T21:09:23.782+0000 7fdb37cc6700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:24.342 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:23.914+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:24.375 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:24.120+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:24.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:24 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:24.342+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:24.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:24 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:24.375+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:25.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:25.018+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:25.371 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:24 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:24.937+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:25.498 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:25 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:25.163+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:25.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:25 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:25.347+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:25.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:25.371+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:26.212 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:25 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:25.915+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:26.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:25.986+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:26.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:25 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:25.986+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:26.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:26.174+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:26.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:26.337+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:26.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:26.395+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:27.121 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:26 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:26.940+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:27.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:26 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:26.955+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:27.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:26 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:26.955+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:27.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:27.198+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:27.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:27 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:27.360+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:27.872 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:27 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:27.372+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:27.872 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:27 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:27.372+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:28.264 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:27.913+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:28.264 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:27 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:27.913+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:28.339 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:27 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:27.922+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:28.498 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:28.183+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:28.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:28.183+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:28.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:28.367+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:28.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:28 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:28.367+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:28.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:28 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:28.339+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:28.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:28 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:28.339+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:29.241 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:28.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:29.241 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:28 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:28.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:29.367 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:28 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:28.964+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:29.367 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:28 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:28.964+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:29.498 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:29 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:29.198+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:29.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:29 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:29.198+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:29.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:29 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:29.403+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:29.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:29 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:29.403+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:29.525 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:29 smithi097 podman[69553]: 2024-04-25 21:09:29.24155123 +0000 UTC m=+5.558013752 container died f3812fd16936624a3f0fee6cbaa460bea240915685b47f38a35f9ae467233394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-25T21:09:29.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:29 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:29.528+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:29.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:29 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:29.367+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:29.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:29 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:29.367+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:29.862 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:29 smithi097 podman[69553]: 2024-04-25 21:09:29.633845036 +0000 UTC m=+5.950307556 container cleanup f3812fd16936624a3f0fee6cbaa460bea240915685b47f38a35f9ae467233394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T21:09:29.863 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:29 smithi097 bash[69553]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4 2024-04-25T21:09:30.193 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:29 smithi097 podman[69582]: 2024-04-25 21:09:29.868275111 +0000 UTC m=+0.622617632 container remove f3812fd16936624a3f0fee6cbaa460bea240915685b47f38a35f9ae467233394 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:09:30.193 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:29 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:29.915+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:30.194 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:29 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:29.915+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:30.372 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:29 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:29.935+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:30.372 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:29 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:29.935+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:30.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:30 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:30.235+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:30.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:30 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:30.235+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:30.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:30 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:30.420+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:30.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:30 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:30.420+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:30.504 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:30 smithi097 podman[69672]: 2024-04-25 21:09:30.097362894 +0000 UTC m=+0.018472077 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:30.504 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:30 smithi097 podman[69672]: 2024-04-25 21:09:30.194210289 +0000 UTC m=+0.115319465 container create a05be74ba6d6a78b3d413a929b46effdc543ed9994a32922f9ea82ef9b47023e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:09:30.504 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:30 smithi097 podman[69672]: 2024-04-25 21:09:30.438352102 +0000 UTC m=+0.359461274 container init a05be74ba6d6a78b3d413a929b46effdc543ed9994a32922f9ea82ef9b47023e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-25T21:09:30.504 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:30 smithi097 podman[69672]: 2024-04-25 21:09:30.441520819 +0000 UTC m=+0.362629996 container start a05be74ba6d6a78b3d413a929b46effdc543ed9994a32922f9ea82ef9b47023e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-25T21:09:30.641 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:30 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:30.374+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:30.641 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:30 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:30.374+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:30.641 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:30 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:30.495+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:30.863 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:30 smithi097 podman[69672]: 2024-04-25 21:09:30.505040304 +0000 UTC m=+0.426149476 container attach a05be74ba6d6a78b3d413a929b46effdc543ed9994a32922f9ea82ef9b47023e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T21:09:30.863 INFO:journalctl@ceph.osd.4.smithi097.stdout:Apr 25 21:09:30 smithi097 podman[69672]: 2024-04-25 21:09:30.733424748 +0000 UTC m=+0.654533922 container died a05be74ba6d6a78b3d413a929b46effdc543ed9994a32922f9ea82ef9b47023e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T21:09:31.299 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:30 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:30.916+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:31.299 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:30 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:30.916+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:31.318 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.4.service' 2024-04-25T21:09:31.329 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:30 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:30.978+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:31.329 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:30 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:30.978+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:31.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:31.357 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-25T21:09:31.357 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-25T21:09:31.358 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.5 2024-04-25T21:09:31.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:31.244+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:31.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:31.244+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:31.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:31.398+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:31.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:31.398+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:31.592 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:31 smithi139 systemd[1]: Stopping Ceph osd.5 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:31.593 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:31.484+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:31.593 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:31.329+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:31.593 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:31.329+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:31.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:31.380+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:31.872 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:31.707+0000 7f672af2c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:31.872 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:31.707+0000 7f672af2c700 -1 osd.5 106 *** Got signal Terminated *** 2024-04-25T21:09:31.872 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:31 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:31.707+0000 7f672af2c700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:32.291 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:32.023+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:32.291 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:32.023+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:32.336 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:31.886+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:32.336 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:31 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:31.886+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:32.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:32 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:32.336+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:32.621 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:32.521+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:32.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:32.291+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:32.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:32 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:32.291+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:32.749 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:32 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:32.291+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:32.749 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:32 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:32.291+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:32.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:32 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:32.359+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:32.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:32 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:32.359+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:33.288 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:33.007+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:33.288 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:33.007+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:33.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:32 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:32.923+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:33.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:32 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:32.923+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:33.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:33.246+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:33.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:33.246+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:33.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:33.397+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:33.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:33 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:33.397+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:33.551 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:33.288+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:33.551 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:33.288+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:33.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:33.357+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:33.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:33 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:33.551+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:34.293 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:34.030+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:34.294 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:34.030+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:34.294 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:34.030+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.512330+0000 front 2024-04-25T21:09:11.512369+0000 (oldest deadline 2024-04-25T21:09:33.812488+0000) 2024-04-25T21:09:34.332 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:33.959+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:34.333 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:33.959+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:34.333 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:33 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:33.959+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:34.546 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:34.293+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:34.547 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:34.293+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:34.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:34.332+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:34.749 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:34.272+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:34.749 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:34.272+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:34.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:34.383+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:34.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:34.383+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:34.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:34.383+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:34.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:34 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:34.546+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:35.308 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:34.928+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:35.308 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:34.928+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:35.308 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:34 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:34.928+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:35.342 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:35.070+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:35.342 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:35.070+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:35.343 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:35.070+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.512330+0000 front 2024-04-25T21:09:11.512369+0000 (oldest deadline 2024-04-25T21:09:33.812488+0000) 2024-04-25T21:09:35.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:35.308+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:35.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:35.308+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:35.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:35.539+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:35.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:35.539+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:35.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:35.342+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:35.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:35.342+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:35.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:35 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:35.342+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:35.748 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:35.277+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:35.749 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:35.277+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:35.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:35.386+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:35.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:35.386+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:35.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:35 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:35.386+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:36.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:35.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:36.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:35.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:36.358 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:35 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:35.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:36.372 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:36.076+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.208708+0000 front 2024-04-25T21:08:54.208720+0000 (oldest deadline 2024-04-25T21:09:18.908513+0000) 2024-04-25T21:09:36.372 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:36.076+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.610750+0000 front 2024-04-25T21:09:04.610761+0000 (oldest deadline 2024-04-25T21:09:28.611280+0000) 2024-04-25T21:09:36.372 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5[44648]: 2024-04-25T21:09:36.076+0000 7f6727520700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.512330+0000 front 2024-04-25T21:09:11.512369+0000 (oldest deadline 2024-04-25T21:09:33.812488+0000) 2024-04-25T21:09:36.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:36.305+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:36.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:36.305+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:36.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:36.305+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:36.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:36.357+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:36.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:36.357+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:36.748 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:36.262+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:36.749 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:36.262+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:36.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:36.337+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:36.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:36.337+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:36.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:36 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:36.337+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:36.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:36.587+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:36.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:36 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:36.587+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:37.337 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:36.944+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:37.337 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:36.944+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:37.337 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:36 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:36.944+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:37.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:37.230+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:37.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:37.230+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:37.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:37.344+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:37.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:37.344+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:37.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:37 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:37.344+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:37.525 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:37 smithi139 podman[70428]: 2024-04-25 21:09:37.25911138 +0000 UTC m=+5.704991556 container died 3586821e4d902e779e4355feb765ff3b7f78367c5a4f4a5e138a97202dd2f71f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-25T21:09:37.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:37.288+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:37.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:37.288+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:37.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:37.288+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:37.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:37.337+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:37.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:37.337+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:37.872 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:37 smithi139 podman[70428]: 2024-04-25 21:09:37.624026565 +0000 UTC m=+6.069906751 container cleanup 3586821e4d902e779e4355feb765ff3b7f78367c5a4f4a5e138a97202dd2f71f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:09:37.872 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:37 smithi139 bash[70428]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5 2024-04-25T21:09:37.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:37.552+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:37.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:37 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:37.552+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:38.236 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:37 smithi139 podman[70448]: 2024-04-25 21:09:37.875178571 +0000 UTC m=+0.619664597 container remove 3586821e4d902e779e4355feb765ff3b7f78367c5a4f4a5e138a97202dd2f71f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:09:38.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:37.933+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:38.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:37.933+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:38.344 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:37 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:37.933+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:38.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:38.378+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:38.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:38.378+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:38.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:38.378+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:38.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:38.231+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:38.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:38.231+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:38.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:38 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:38.231+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:38.523 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:38 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:38.280+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:38.523 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:38 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:38.280+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:38.523 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:38 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:38.280+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:38.523 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:38 smithi139 podman[70535]: 2024-04-25 21:09:38.140417123 +0000 UTC m=+0.018095461 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:38.524 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:38 smithi139 podman[70535]: 2024-04-25 21:09:38.276386695 +0000 UTC m=+0.154065026 container create b09e404fd77e6af4874f29b5a281f642347dfa75bfce956f4c8bded4b83a7169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True) 2024-04-25T21:09:38.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:38 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:38.343+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:38.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:38 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:38.343+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:38.856 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:38 smithi139 podman[70535]: 2024-04-25 21:09:38.562307207 +0000 UTC m=+0.439985536 container init b09e404fd77e6af4874f29b5a281f642347dfa75bfce956f4c8bded4b83a7169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-25T21:09:38.856 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:38 smithi139 podman[70535]: 2024-04-25 21:09:38.565397269 +0000 UTC m=+0.443075600 container start b09e404fd77e6af4874f29b5a281f642347dfa75bfce956f4c8bded4b83a7169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-04-25T21:09:38.856 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:38 smithi139 podman[70535]: 2024-04-25 21:09:38.645594226 +0000 UTC m=+0.523272562 container attach b09e404fd77e6af4874f29b5a281f642347dfa75bfce956f4c8bded4b83a7169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-25T21:09:38.856 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:38 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:38.523+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:38.856 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:38 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:38.523+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:39.122 INFO:journalctl@ceph.osd.5.smithi139.stdout:Apr 25 21:09:38 smithi139 podman[70535]: 2024-04-25 21:09:38.85650317 +0000 UTC m=+0.734181498 container died b09e404fd77e6af4874f29b5a281f642347dfa75bfce956f4c8bded4b83a7169 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:09:39.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:38 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:38.980+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:39.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:38 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:38.980+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:39.357 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:38 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:38.980+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:39.431 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:39 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:39.310+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:39.431 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:39 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:39.310+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:39.431 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:39 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:39.310+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:39.450 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.5.service' 2024-04-25T21:09:39.490 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:39.490 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-25T21:09:39.490 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-25T21:09:39.490 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.6 2024-04-25T21:09:39.498 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:39.186+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:39.498 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:39.186+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:39.498 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:39.186+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:39.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:39.368+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:39.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:39.368+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:39.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:39.368+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:39.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:39.356+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:39.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:39.356+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:39.790 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 systemd[1]: Stopping Ceph osd.6 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:39.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:39 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:39.528+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:39.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:39 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:39.528+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:40.174 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:39.790+0000 7fac42194700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:40.174 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:39.790+0000 7fac42194700 -1 osd.6 106 *** Got signal Terminated *** 2024-04-25T21:09:40.174 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:39.790+0000 7fac42194700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:40.174 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:40.173+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:40.174 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:40.173+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:40.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:39.933+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:40.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:39.933+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:40.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:39 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:39.933+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:40.434 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:40.173+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:40.435 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:40.351+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:40.435 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:40.351+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:40.435 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:40 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:40.351+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:40.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:40 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:40.509+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:40.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:40 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:40.509+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:40.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:40 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:40.300+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:40.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:40 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:40.300+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:40.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:40 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:40.300+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:40.862 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:40 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:40.385+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:40.863 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:40 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:40.385+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:41.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:40 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:40.980+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:41.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:40 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:40.980+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:41.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:40 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:40.980+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:41.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:41 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:41.293+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:41.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:41 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:41.293+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:41.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:41 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:41.293+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:41.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:41.150+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:41.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:41.150+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:41.499 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:41.150+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:41.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:41.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:41.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:41.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:41.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:41 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:41.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:41.862 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:41 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:41.419+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:41.863 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:41 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:41.419+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:41.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:41 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:41.485+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:41.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:41 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:41.485+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:42.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:41 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:41.950+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:42.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:41 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:41.950+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:42.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:41 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:41.950+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:42.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:41 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:41.950+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:42.397 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:42.112+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:42.398 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:42.112+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:42.398 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:42.112+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:42.505 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:42 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:42.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:42.505 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:42 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:42.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:42.505 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:42 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:42.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:42.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:42.398+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:42.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:42.398+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:42.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:42.398+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:42.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:42 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:42.398+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:42.862 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:42.408+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:42.863 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:42.408+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:42.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:42 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:42.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:42.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:42 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:42.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:42.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:42 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:42.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:43.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:42.989+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:43.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:42.989+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:43.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:42.989+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:43.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:42.989+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:43.429 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:43.073+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:43.429 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:43.073+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:43.430 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:43.073+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:43.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:43 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:43.361+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:43.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:43 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:43.361+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:43.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:43 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:43.244+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:43.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:43 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:43.244+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:43.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:43 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:43.244+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:43.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:43 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:43.470+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:43.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:43 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:43.470+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:43.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:43 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:43.470+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:43.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:43.429+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:43.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:43.429+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:43.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:43.429+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:43.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:43 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:43.429+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:44.349 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:43 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:43.979+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:44.349 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:43 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:43.979+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:44.349 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:43 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:43.979+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:44.349 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:43 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:43.979+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:44.407 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:44.041+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.441695+0000 front 2024-04-25T21:08:56.441786+0000 (oldest deadline 2024-04-25T21:09:21.741935+0000) 2024-04-25T21:09:44.407 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:44.041+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.642872+0000 front 2024-04-25T21:09:04.642803+0000 (oldest deadline 2024-04-25T21:09:28.142894+0000) 2024-04-25T21:09:44.408 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6[56983]: 2024-04-25T21:09:44.041+0000 7fac3a780700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.244253+0000 front 2024-04-25T21:09:12.244199+0000 (oldest deadline 2024-04-25T21:09:37.544158+0000) 2024-04-25T21:09:44.515 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:44.213+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:44.515 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:44.213+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:44.515 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:44.213+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:44.515 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:44.213+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:44.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:44 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:44.349+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:44.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:44 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:44.349+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:44.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:44.407+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:44.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:44.407+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:44.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:44.407+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:44.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:44 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:44.407+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:44.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:44.514+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:44.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:44.514+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:44.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:44 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:44.514+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:45.339 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:44 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:44.934+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:45.340 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:44 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:44.934+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:45.340 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:44 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:44.934+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:45.340 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:44 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:44.934+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:45.561 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:45.192+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:45.562 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:45.192+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:45.562 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:45.192+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:45.562 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:45.192+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:45.605 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:45 smithi096 podman[116412]: 2024-04-25 21:09:45.276732192 +0000 UTC m=+5.595730170 container died caaadcdfd98326e4ea7eaa1b1a8e6c9519826c5a94e086fbf4d9448b623a4bbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-25T21:09:45.606 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:45 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:45.441+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:45.606 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:45 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:45.441+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:45.606 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:45 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:45.441+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:45.606 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:45 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:45.441+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:45.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:45.340+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:45.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:45.340+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:45.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:45.340+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:45.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:45.561+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:45.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:45.561+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:45.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:45 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:45.561+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:45.998 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:45 smithi096 podman[116412]: 2024-04-25 21:09:45.606566848 +0000 UTC m=+5.925564824 container cleanup caaadcdfd98326e4ea7eaa1b1a8e6c9519826c5a94e086fbf4d9448b623a4bbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T21:09:45.999 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:45 smithi096 bash[116412]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6 2024-04-25T21:09:45.999 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:45 smithi096 podman[116441]: 2024-04-25 21:09:45.832640308 +0000 UTC m=+0.549688645 container remove caaadcdfd98326e4ea7eaa1b1a8e6c9519826c5a94e086fbf4d9448b623a4bbd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-25T21:09:46.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:45.912+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:46.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:45.912+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:46.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:45.912+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:46.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:45 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:45.912+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:46.549 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:46.191+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:46.549 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:46.191+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:46.549 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:46.191+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:46.549 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:46.191+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:46.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:46.342+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:46.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:46.342+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:46.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:46.342+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:46.618 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:46 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:46.489+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:46.618 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:46 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:46.489+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:46.618 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:46 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:46.489+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:46.619 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:46 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:46.489+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:46.619 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:46 smithi096 podman[116533]: 2024-04-25 21:09:46.185324071 +0000 UTC m=+0.018104573 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:46.619 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:46 smithi096 podman[116533]: 2024-04-25 21:09:46.300193007 +0000 UTC m=+0.132973493 container create 80ebd298f8f4837c6bf19532e3e8d9a0146695c6e0ec93527a2ce43accecf152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6-deactivate, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-25T21:09:46.621 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:46 smithi096 podman[116533]: 2024-04-25 21:09:46.527267777 +0000 UTC m=+0.360048264 container init 80ebd298f8f4837c6bf19532e3e8d9a0146695c6e0ec93527a2ce43accecf152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-25T21:09:46.621 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:46 smithi096 podman[116533]: 2024-04-25 21:09:46.530405574 +0000 UTC m=+0.363186064 container start 80ebd298f8f4837c6bf19532e3e8d9a0146695c6e0ec93527a2ce43accecf152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:09:46.621 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:46 smithi096 podman[116533]: 2024-04-25 21:09:46.618963848 +0000 UTC m=+0.451744353 container attach 80ebd298f8f4837c6bf19532e3e8d9a0146695c6e0ec93527a2ce43accecf152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T21:09:46.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:46.548+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:46.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:46.548+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:46.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:46 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:46.548+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:46.999 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:46 smithi096 podman[116533]: 2024-04-25 21:09:46.82058028 +0000 UTC m=+0.653360768 container died 80ebd298f8f4837c6bf19532e3e8d9a0146695c6e0ec93527a2ce43accecf152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:09:47.321 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:46.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:47.321 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:46.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:47.321 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:46.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:47.322 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:46 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:46.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:47.352 INFO:journalctl@ceph.osd.6.smithi096.stdout:Apr 25 21:09:47 smithi096 podman[116558]: 2024-04-25 21:09:47.339486091 +0000 UTC m=+0.513300127 container remove 80ebd298f8f4837c6bf19532e3e8d9a0146695c6e0ec93527a2ce43accecf152 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:09:47.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.6.service' 2024-04-25T21:09:47.437 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:47.437 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-25T21:09:47.437 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-25T21:09:47.438 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.7 2024-04-25T21:09:47.565 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:47.201+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:47.565 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:47.201+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:47.565 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:47.201+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:47.565 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:47.201+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:47.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:47.321+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:47.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:47.321+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:47.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:47.321+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:47.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 systemd[1]: Stopping Ceph osd.7 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:47.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:47.527+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:47.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:47.527+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:47.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:47.527+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:47.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:47 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:47.527+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:47.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:47.565+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:47.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:47.565+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:47.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:47 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:47.565+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:48.112 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:47.727+0000 7fc835e46700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:48.113 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:47.727+0000 7fc835e46700 -1 osd.7 106 *** Got signal Terminated *** 2024-04-25T21:09:48.113 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:47.727+0000 7fc835e46700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:48.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:47.920+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:48.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:47.920+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:48.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:47.920+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:48.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:47.920+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:48.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:48.250+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:48.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:48.250+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:48.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:48.250+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:48.525 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:48.250+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:48.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:48.310+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:48.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:48.310+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:48.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:48.310+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:48.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:48.525+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:48.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:48.525+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:48.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:48 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:48.525+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:48.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:48 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:48.529+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:48.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:48 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:48.529+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:48.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:48 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:48.529+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:48.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:48 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:48.529+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:49.310 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:48.917+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:49.310 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:48.917+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:49.310 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:48.917+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:49.310 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:48 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:48.917+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:49.506 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:49.229+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:49.506 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:49.229+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:49.506 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:49.229+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:49.506 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:49.229+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:49.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:49.309+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:49.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:49.309+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:49.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:49.309+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:49.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:49.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:49.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:49.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:49.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:49.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:49.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:49 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:49.505+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:49.998 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:49 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:49.533+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:49.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:49 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:49.533+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:49.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:49 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:49.533+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:49.999 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:49 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:49.533+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:50.312 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:49.894+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:50.312 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:49.894+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:50.312 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:49.894+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:50.313 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:49 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:49.894+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:50.512 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:50.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:50.512 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:50.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:50.512 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:50.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:50.512 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:50.251+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:50.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:50.312+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:50.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:50.312+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:50.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:50.312+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:50.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:50.312+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.210065+0000 front 2024-04-25T21:09:25.210109+0000 (oldest deadline 2024-04-25T21:09:49.910183+0000) 2024-04-25T21:09:50.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:50 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:50.487+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:50.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:50 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:50.487+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:50.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:50 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:50.487+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:50.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:50 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:50.487+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:50.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:50.511+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:50.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:50.511+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:50.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:50.511+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:50.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:50 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:50.511+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:51.335 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:50.935+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:51.335 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:50.935+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:51.335 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:50.935+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:51.335 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:50 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:50.935+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:51.529 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:51.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:51.529 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:51.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:51.529 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:51.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:51.529 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:51.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:51.529 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:51.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:51.612 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:51.335+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:51.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:51.335+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:51.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:51.335+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:51.613 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:51.335+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.210065+0000 front 2024-04-25T21:09:25.210109+0000 (oldest deadline 2024-04-25T21:09:49.910183+0000) 2024-04-25T21:09:51.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:51.451+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:51.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:51.451+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:51.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:51.451+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:51.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:51.451+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:51.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:51 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:51.451+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:51.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:51.529+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:51.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:51.529+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:51.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:51.529+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:51.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:51 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:51.529+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:52.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:51.905+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:52.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:51.905+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:52.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:51.905+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:52.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:51.905+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:52.490 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:52.170+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:52.490 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:52.170+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:52.490 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:52.170+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:52.491 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:52.170+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:52.491 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:52.170+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:52.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:52 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:52.431+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:52.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:52 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:52.431+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:52.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:52 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:52.431+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:52.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:52 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:52.431+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:52.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:52 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:52.431+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:52.862 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:52.374+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:01.806975+0000 front 2024-04-25T21:09:01.807016+0000 (oldest deadline 2024-04-25T21:09:30.607703+0000) 2024-04-25T21:09:52.863 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:52.374+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:11.108428+0000 front 2024-04-25T21:09:11.108489+0000 (oldest deadline 2024-04-25T21:09:35.208572+0000) 2024-04-25T21:09:52.863 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:52.375+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:15.208983+0000 front 2024-04-25T21:09:15.209091+0000 (oldest deadline 2024-04-25T21:09:45.209702+0000) 2024-04-25T21:09:52.863 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7[50785]: 2024-04-25T21:09:52.375+0000 7fc83243a700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.210065+0000 front 2024-04-25T21:09:25.210109+0000 (oldest deadline 2024-04-25T21:09:49.910183+0000) 2024-04-25T21:09:52.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:52.490+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:52.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:52.490+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:52.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:52.490+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:52.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:52 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:52.490+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:53.244 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:53 smithi097 podman[69779]: 2024-04-25 21:09:53.063639951 +0000 UTC m=+5.437301645 container died 245cf46af4dca074b5fe1a817362b60db0ec034bbc883de07294e82390f91f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T21:09:53.244 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:52.936+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:53.244 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:52.936+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:53.244 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:52.936+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:53.244 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:52 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:52.936+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:53.480 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:53.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:53.480 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:53.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:53.480 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:53.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:53.480 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:53.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:53.480 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:53.203+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:53.494 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:53 smithi097 podman[69779]: 2024-04-25 21:09:53.335868545 +0000 UTC m=+5.709530237 container cleanup 245cf46af4dca074b5fe1a817362b60db0ec034bbc883de07294e82390f91f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-25T21:09:53.494 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:53 smithi097 bash[69779]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7 2024-04-25T21:09:53.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:53.411+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:53.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:53.411+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:53.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:53.411+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:53.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:53.411+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:53.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:53 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:53.411+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:53.862 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:53 smithi097 podman[69796]: 2024-04-25 21:09:53.495512093 +0000 UTC m=+0.428875726 container remove 245cf46af4dca074b5fe1a817362b60db0ec034bbc883de07294e82390f91f67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-04-25T21:09:53.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:53.480+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:53.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:53.480+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:53.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:53.480+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:53.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:53 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:53.480+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:54.156 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:53 smithi097 podman[69887]: 2024-04-25 21:09:53.80099083 +0000 UTC m=+0.017765004 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:09:54.156 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:53 smithi097 podman[69887]: 2024-04-25 21:09:53.913079617 +0000 UTC m=+0.129853822 container create 820b6ef987da8b3ae8c8d3bc9b11ff2cd5b402548eef2d40bb20512e2211d562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:09:54.156 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:53 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:53.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:54.156 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:53 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:53.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:54.156 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:53 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:53.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:54.157 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:53 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:53.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:54.157 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:53 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:53.922+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:09:54.434 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:54.166+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:54.435 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:54.166+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:54.435 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:54.166+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:54.435 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:54.166+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:54.435 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:54.166+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:54.447 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 podman[69887]: 2024-04-25 21:09:54.157053774 +0000 UTC m=+0.373827936 container init 820b6ef987da8b3ae8c8d3bc9b11ff2cd5b402548eef2d40bb20512e2211d562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-25T21:09:54.447 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 podman[69887]: 2024-04-25 21:09:54.160163398 +0000 UTC m=+0.376937563 container start 820b6ef987da8b3ae8c8d3bc9b11ff2cd5b402548eef2d40bb20512e2211d562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-04-25T21:09:54.447 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 podman[69887]: 2024-04-25 21:09:54.24034187 +0000 UTC m=+0.457116042 container attach 820b6ef987da8b3ae8c8d3bc9b11ff2cd5b402548eef2d40bb20512e2211d562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416) 2024-04-25T21:09:54.734 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 podman[69887]: 2024-04-25 21:09:54.448243648 +0000 UTC m=+0.665017805 container died 820b6ef987da8b3ae8c8d3bc9b11ff2cd5b402548eef2d40bb20512e2211d562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-04-25T21:09:54.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:54 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:54.387+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:54.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:54 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:54.387+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:54.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:54 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:54.387+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:54.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:54 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:54.387+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:54.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:54 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:54.387+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:54.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:54.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:54.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:54.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:54.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:54.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:54.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:54 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:54.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:54.960 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.7.service' 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 podman[69913]: 2024-04-25 21:09:54.935081045 +0000 UTC m=+0.480970778 container remove 820b6ef987da8b3ae8c8d3bc9b11ff2cd5b402548eef2d40bb20512e2211d562 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 systemd[1]: ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.7.service: Deactivated successfully. 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 systemd[1]: Stopped Ceph osd.7 for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.7.smithi097.stdout:Apr 25 21:09:54 smithi097 systemd[1]: ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.7.service: Consumed 51.188s CPU time. 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:54 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:54.885+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:54 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:54.885+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:54 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:54.885+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:54 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:54.885+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:54.985 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:54 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:54.885+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:09:54.997 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:09:54.998 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-25T21:09:54.998 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-25T21:09:54.998 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.8 2024-04-25T21:09:55.299 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 systemd[1]: Stopping Ceph osd.8 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:09:55.300 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:55.196+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:55.300 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:55.196+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:55.300 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:55.196+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:55.300 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:55.196+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:55.300 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:55.196+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:55.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.299+0000 7f6aba822700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:09:55.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.299+0000 7f6aba822700 -1 osd.8 106 *** Got signal Terminated *** 2024-04-25T21:09:55.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.299+0000 7f6aba822700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:09:55.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:55.623 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:55.623 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:55.623 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:55 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:55.434+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:55.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:55 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:55.362+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:55.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:55 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:55.362+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:55.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:55 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:55.362+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:55.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:55 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:55.362+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:55.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:55 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:55.362+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:56.112 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:55.848+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:56.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:55.848+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:56.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:55.848+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:56.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:55.848+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:56.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:55.848+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:09:56.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:56.401+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:56.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:56.401+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:56.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:56.401+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:56.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:56.401+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:56.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:56.161+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:56.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:56.161+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:56.623 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:56.161+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:56.623 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:56.161+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:56.623 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:56 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:56.161+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:56.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:56 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:56.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:56.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:56 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:56.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:56.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:56 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:56.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:56.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:56 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:56.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:56.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:56 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:56.379+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:57.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:56.869+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:57.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:56.869+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:57.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:56.869+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:57.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:56.869+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:57.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:56 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:56.869+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:09:57.407 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:57.152+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:57.407 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:57.152+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:57.407 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:57.152+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:57.407 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:57.152+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:57.407 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:57.152+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:57.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:57 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:57.399+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:57.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:57 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:57.399+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:57.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:57 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:57.399+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:57.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:57 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:57.399+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:57.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:57 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:57.399+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:57.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:57 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:57.399+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:09:57.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:57.408+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:57.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:57.408+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:57.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:57.408+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:57.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:57.408+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:58.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:57 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:57.856+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:58.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:57 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:57.856+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:58.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:57 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:57.856+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:58.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:57 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:57.856+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:58.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:57 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:57.856+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:09:58.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:58.118+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:58.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:58.118+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:58.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:58.118+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:58.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:58.118+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:58.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:58.118+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:58.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:58.118+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:09:58.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:58 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:58.361+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:58.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:58 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:58.361+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:58.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:58 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:58.361+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:58.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:58 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:58.361+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:58.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:58 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:58.361+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:58.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:58 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:58.361+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:09:58.871 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:58.423+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:58.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:58.423+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:58.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:58.423+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:58.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:58 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:58.423+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:59.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:58 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:58.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:09:59.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:58 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:58.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:09:59.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:58 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:58.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:09:59.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:58 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:58.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:09:59.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:58 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:58.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:09:59.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:58 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:58.876+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:09:59.447 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:59.127+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:09:59.447 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:59.127+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:09:59.447 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:59.127+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:09:59.447 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:59.127+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:09:59.447 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:59.127+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:09:59.447 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:09:59.127+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:09:59.720 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:59.447+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.324273+0000 front 2024-04-25T21:09:04.324470+0000 (oldest deadline 2024-04-25T21:09:29.024354+0000) 2024-04-25T21:09:59.720 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:59.447+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.024792+0000 front 2024-04-25T21:09:09.024765+0000 (oldest deadline 2024-04-25T21:09:34.924914+0000) 2024-04-25T21:09:59.720 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:59.447+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.927025+0000 front 2024-04-25T21:09:18.926902+0000 (oldest deadline 2024-04-25T21:09:41.826783+0000) 2024-04-25T21:09:59.720 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:59.447+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:25.328132+0000 front 2024-04-25T21:09:25.328085+0000 (oldest deadline 2024-04-25T21:09:49.328366+0000) 2024-04-25T21:09:59.720 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:09:59 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8[51216]: 2024-04-25T21:09:59.447+0000 7f6ab2e0e700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:36.329674+0000 front 2024-04-25T21:09:36.329599+0000 (oldest deadline 2024-04-25T21:09:59.229682+0000) 2024-04-25T21:09:59.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:59.352+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:09:59.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:59.352+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:09:59.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:59.352+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:09:59.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:59.352+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:09:59.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:59.352+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:09:59.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:09:59 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:09:59.352+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:00.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:59 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:59.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:00.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:59 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:59.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:00.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:59 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:59.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:00.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:59 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:59.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:00.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:59 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:59.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:00.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:09:59 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:09:59.925+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:00.601 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:00 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:00.131+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:00.601 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:00 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:00.131+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:00.601 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:00 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:00.131+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:00.601 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:00 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:00.131+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:00.602 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:00 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:00.131+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:00.602 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:00 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:00.131+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:00.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:00.373+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:00.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:00.373+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:00.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:00.373+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:00.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:00.373+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:00.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:00.373+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:00.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:00 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:00.373+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:00.872 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:00 smithi139 podman[70642]: 2024-04-25 21:10:00.679950028 +0000 UTC m=+5.479493863 container died 4b896d54b25ba9cb32ff3e929747b2597206e9461645baa8b4cfa7b7b275fffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:10:01.260 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 podman[70642]: 2024-04-25 21:10:01.067652615 +0000 UTC m=+5.867196491 container cleanup 4b896d54b25ba9cb32ff3e929747b2597206e9461645baa8b4cfa7b7b275fffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-25T21:10:01.260 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 bash[70642]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8 2024-04-25T21:10:01.261 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:01 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:01.176+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:01.261 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:01 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:01.176+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:01.261 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:01 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:01.176+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:01.261 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:01 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:01.176+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:01.261 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:01 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:01.176+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:01.261 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:01 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:01.176+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:01.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:00.947+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:01.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:00.947+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:01.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:00.947+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:01.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:00.947+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:01.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:00.947+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:01.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:00 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:00.947+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:01.622 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 podman[70672]: 2024-04-25 21:10:01.26110601 +0000 UTC m=+0.570757331 container remove 4b896d54b25ba9cb32ff3e929747b2597206e9461645baa8b4cfa7b7b275fffa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-25T21:10:01.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:01 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:01.381+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:01.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:01 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:01.381+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:01.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:01 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:01.381+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:01.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:01 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:01.381+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:01.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:01 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:01.381+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:01.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:01 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:01.381+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:02.014 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 podman[70763]: 2024-04-25 21:10:01.582843844 +0000 UTC m=+0.018406767 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:10:02.014 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 podman[70763]: 2024-04-25 21:10:01.686949183 +0000 UTC m=+0.122512096 container create 83ac19a27f4c873b87ba3372040fb44a0564968fa52de0acb0e1005d6634eecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:10:02.014 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 podman[70763]: 2024-04-25 21:10:01.92239074 +0000 UTC m=+0.357953665 container init 83ac19a27f4c873b87ba3372040fb44a0564968fa52de0acb0e1005d6634eecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T21:10:02.014 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:01 smithi139 podman[70763]: 2024-04-25 21:10:01.925438419 +0000 UTC m=+0.361001332 container start 83ac19a27f4c873b87ba3372040fb44a0564968fa52de0acb0e1005d6634eecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-25T21:10:02.014 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:02 smithi139 podman[70763]: 2024-04-25 21:10:02.014298112 +0000 UTC m=+0.449861027 container attach 83ac19a27f4c873b87ba3372040fb44a0564968fa52de0acb0e1005d6634eecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-25T21:10:02.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:01 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:01.974+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:02.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:01 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:01.974+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:02.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:01 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:01.974+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:02.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:01 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:01.974+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:02.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:01 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:01.974+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:02.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:01 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:01.974+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:02.372 INFO:journalctl@ceph.osd.8.smithi139.stdout:Apr 25 21:10:02 smithi139 podman[70763]: 2024-04-25 21:10:02.222512456 +0000 UTC m=+0.658075359 container died 83ac19a27f4c873b87ba3372040fb44a0564968fa52de0acb0e1005d6634eecd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:10:02.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:02.165+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:02.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:02.165+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:02.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:02.165+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:02.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:02.165+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:02.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:02.165+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:02.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:02.165+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:02.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:02.404+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:02.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:02.404+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:02.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:02.404+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:02.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:02.404+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:02.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:02.404+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:02.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:02.404+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:02.815 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.8.service' 2024-04-25T21:10:02.856 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:02.857 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-25T21:10:02.857 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-25T21:10:02.857 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.9 2024-04-25T21:10:03.156 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:02 smithi096 systemd[1]: Stopping Ceph osd.9 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:03.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:02 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:02.960+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:03.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:02 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:02.960+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:03.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:02 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:02.960+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:03.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:02 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:02.960+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:03.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:02 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:02.960+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:03.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:02 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:02.960+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.156+0000 7f0aee2f1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.156+0000 7f0aee2f1700 -1 osd.9 106 *** Got signal Terminated *** 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.156+0000 7f0aee2f1700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.370+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.370+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.370+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.370+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.370+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:03.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:03 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:03.370+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:03.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:03.141+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:03.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:03.141+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:03.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:03.141+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:03.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:03.141+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:03.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:03.141+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:03.623 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:03 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:03.141+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:04.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:03 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:03.938+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:04.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:03 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:03.938+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:04.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:03 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:03.938+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:04.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:03 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:03.938+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:04.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:03 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:03.938+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:04.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:03 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:03.938+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:04.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:04 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:04.115+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:04.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:04 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:04.115+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:04.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:04 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:04.115+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:04.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:04 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:04.115+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:04.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:04 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:04.115+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:04.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:04 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:04.115+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:04.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:04.331+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:04.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:04.331+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:04.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:04.331+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:04.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:04.331+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:04.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:04.331+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:04.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:04 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:04.331+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:05.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:04 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:04.969+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:05.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:04 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:04.969+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:05.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:04 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:04.969+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:05.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:04 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:04.969+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:05.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:04 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:04.969+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:05.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:04 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:04.969+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:05.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:05.099+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:05.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:05.099+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:05.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:05.099+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:05.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:05.099+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:05.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:05.099+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:05.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:05 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:05.099+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:05.748 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:05.311+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:05.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:05.311+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:05.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:05.311+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:05.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:05.311+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:05.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:05.311+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:05.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:05 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:05.311+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:06.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:05 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:05.966+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:06.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:05 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:05.966+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:06.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:05 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:05.966+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:06.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:05 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:05.966+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:06.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:05 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:05.966+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:06.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:05 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:05.966+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:06.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:06 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:06.051+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:06.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:06 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:06.051+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:06.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:06 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:06.051+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:06.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:06 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:06.051+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:06.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:06 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:06.051+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:06.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:06 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:06.051+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:06.749 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:06 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:06.274+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:43.085225+0000 front 2024-04-25T21:09:43.085277+0000 (oldest deadline 2024-04-25T21:10:05.385478+0000) 2024-04-25T21:10:07.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:06 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:06.931+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:07.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:06 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:06.931+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:07.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:06 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:06.931+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:07.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:06 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:06.931+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:07.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:06 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:06.931+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:07.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:06 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:06.931+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:07.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.009+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:07.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.009+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:07.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.009+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:07.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.009+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:07.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.009+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:07.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.009+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:54.674315+0000 front 2024-04-25T21:08:54.674506+0000 (oldest deadline 2024-04-25T21:09:20.574389+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:00.575166+0000 front 2024-04-25T21:09:00.575144+0000 (oldest deadline 2024-04-25T21:09:27.575674+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:10.278234+0000 front 2024-04-25T21:09:10.278047+0000 (oldest deadline 2024-04-25T21:09:34.378133+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.779632+0000 front 2024-04-25T21:09:20.779753+0000 (oldest deadline 2024-04-25T21:09:41.879724+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:24.681591+0000 front 2024-04-25T21:09:24.681769+0000 (oldest deadline 2024-04-25T21:09:50.581458+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.483386+0000 front 2024-04-25T21:09:34.483403+0000 (oldest deadline 2024-04-25T21:09:57.383408+0000) 2024-04-25T21:10:07.499 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:07 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9[63600]: 2024-04-25T21:10:07.231+0000 7f0ae68dd700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:43.085225+0000 front 2024-04-25T21:09:43.085277+0000 (oldest deadline 2024-04-25T21:10:05.385478+0000) 2024-04-25T21:10:08.208 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:08.209 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:08.209 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:08.209 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:08.209 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:08.209 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:08.209 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:07 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:07.952+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:08.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.991+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:08.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.991+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:08.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.991+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:08.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.991+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:08.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.991+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:08.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:07.991+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:08.831 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:08 smithi096 podman[116640]: 2024-04-25 21:10:08.561502593 +0000 UTC m=+5.507905448 container died 50b13a2e3a0a697c220963a15826aba1d2a5c7e211e199f6807190b9bf784052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-04-25T21:10:09.206 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:08 smithi096 podman[116640]: 2024-04-25 21:10:08.930924305 +0000 UTC m=+5.877327155 container cleanup 50b13a2e3a0a697c220963a15826aba1d2a5c7e211e199f6807190b9bf784052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-25T21:10:09.206 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:08 smithi096 bash[116640]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:09.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:08 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:08.942+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:09.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:08 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:08.965+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:09.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:08 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:08.965+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:09.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:08 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:08.965+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:09.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:08 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:08.965+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:09.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:08 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:08.965+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:09.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:08 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:08.965+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:09.498 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:09 smithi096 podman[116657]: 2024-04-25 21:10:09.207346356 +0000 UTC m=+0.639082088 container remove 50b13a2e3a0a697c220963a15826aba1d2a5c7e211e199f6807190b9bf784052 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-25T21:10:09.868 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:09 smithi096 podman[116748]: 2024-04-25 21:10:09.452584648 +0000 UTC m=+0.017931150 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:10:09.868 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:09 smithi096 podman[116748]: 2024-04-25 21:10:09.566433782 +0000 UTC m=+0.131780270 container create 514341d16e40faf5db0870c1410eb5c73651bd1b1fb69900eeb3a3038a2c6ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-25T21:10:09.868 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:09 smithi096 podman[116748]: 2024-04-25 21:10:09.785405942 +0000 UTC m=+0.350752434 container init 514341d16e40faf5db0870c1410eb5c73651bd1b1fb69900eeb3a3038a2c6ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:10:09.868 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:09 smithi096 podman[116748]: 2024-04-25 21:10:09.788391607 +0000 UTC m=+0.353738097 container start 514341d16e40faf5db0870c1410eb5c73651bd1b1fb69900eeb3a3038a2c6ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T21:10:09.868 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:09 smithi096 podman[116748]: 2024-04-25 21:10:09.86845283 +0000 UTC m=+0.433799333 container attach 514341d16e40faf5db0870c1410eb5c73651bd1b1fb69900eeb3a3038a2c6ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-25T21:10:10.249 INFO:journalctl@ceph.osd.9.smithi096.stdout:Apr 25 21:10:10 smithi096 podman[116748]: 2024-04-25 21:10:10.07526285 +0000 UTC m=+0.640609346 container died 514341d16e40faf5db0870c1410eb5c73651bd1b1fb69900eeb3a3038a2c6ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:10:10.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:10.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:10.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:10.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:10.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:10.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:10.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:09 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:09.970+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:10.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:09 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:09.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:10.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:09 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:09.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:10.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:09 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:09.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:10.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:09 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:09.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:10.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:09 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:09.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:10.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:09 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:09.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:10.675 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.9.service' 2024-04-25T21:10:10.713 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:10.713 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-25T21:10:10.713 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-25T21:10:10.713 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.10 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 systemd[1]: Stopping Ceph osd.10 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:10 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:10.932+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.029+0000 7f1eb6bcf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:11.030 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.029+0000 7f1eb6bcf700 -1 osd.10 106 *** Got signal Terminated *** 2024-04-25T21:10:11.031 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.029+0000 7f1eb6bcf700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:10:11.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:10 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:10.997+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:11.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:10 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:10.997+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:11.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:10 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:10.997+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:11.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:10 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:10.997+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:11.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:10 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:10.997+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:11.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:10 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:10.997+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:12.280 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:12 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:12.002+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:12.281 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:12 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:12.002+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:12.281 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:12 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:12.002+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:12.281 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:12 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:12.002+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:12.281 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:12 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:12.002+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:12.281 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:12 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:12.002+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:12.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:11 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:11.906+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:13.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:13.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:13.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:13.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:13.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:13.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:13.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:12 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:12.924+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:13.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.014+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:13.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.014+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:13.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.014+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:13.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.014+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:13.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.014+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:13.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.014+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:14.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:14.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:14.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:14.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:14.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:14.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:14.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:13 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:13.875+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:14.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:14.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:14.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:14.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:14.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:14.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:13 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:13.979+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:15.362 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:15.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:15.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:15.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:15.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:15.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:15.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:14 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:14.889+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:15.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:15 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:15.004+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:56.936229+0000 front 2024-04-25T21:08:56.936257+0000 (oldest deadline 2024-04-25T21:09:22.836088+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:02.836455+0000 front 2024-04-25T21:09:02.836526+0000 (oldest deadline 2024-04-25T21:09:25.637440+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:12.039066+0000 front 2024-04-25T21:09:12.039133+0000 (oldest deadline 2024-04-25T21:09:33.739018+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:18.940915+0000 front 2024-04-25T21:09:18.940964+0000 (oldest deadline 2024-04-25T21:09:41.840813+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:28.643016+0000 front 2024-04-25T21:09:28.642996+0000 (oldest deadline 2024-04-25T21:09:53.343057+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.444061+0000 front 2024-04-25T21:09:34.444119+0000 (oldest deadline 2024-04-25T21:09:58.543989+0000) 2024-04-25T21:10:16.363 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:15 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10[57543]: 2024-04-25T21:10:15.897+0000 7f1eaf1bb700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6822 osd.6 since back 2024-04-25T21:09:40.845312+0000 front 2024-04-25T21:09:40.845340+0000 (oldest deadline 2024-04-25T21:10:07.845478+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:16.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:16 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:16.012+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:16.754 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:16 smithi097 podman[70008]: 2024-04-25 21:10:16.490334918 +0000 UTC m=+5.585453842 container died 681efe154313dd2dad6827508c98b6240bfc36f083139c6397e1fa1216e4e473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:10:17.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:16 smithi097 podman[70008]: 2024-04-25 21:10:16.85422331 +0000 UTC m=+5.949342238 container cleanup 681efe154313dd2dad6827508c98b6240bfc36f083139c6397e1fa1216e4e473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240416) 2024-04-25T21:10:17.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:16 smithi097 bash[70008]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:17.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:17 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:17.038+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:17.473 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70025]: 2024-04-25 21:10:17.113778118 +0000 UTC m=+0.622839539 container remove 681efe154313dd2dad6827508c98b6240bfc36f083139c6397e1fa1216e4e473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416) 2024-04-25T21:10:17.749 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70117]: 2024-04-25 21:10:17.378006693 +0000 UTC m=+0.018267717 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:10:17.749 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70117]: 2024-04-25 21:10:17.481158962 +0000 UTC m=+0.121419970 container create 9140ba397d95bd7da9a1480436903107b4cd6daae3a5fd0e357b5e079e410ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-25T21:10:17.750 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70117]: 2024-04-25 21:10:17.675447222 +0000 UTC m=+0.315708230 container init 9140ba397d95bd7da9a1480436903107b4cd6daae3a5fd0e357b5e079e410ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:10:17.750 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70117]: 2024-04-25 21:10:17.678578594 +0000 UTC m=+0.318839606 container start 9140ba397d95bd7da9a1480436903107b4cd6daae3a5fd0e357b5e079e410ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True) 2024-04-25T21:10:17.750 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70117]: 2024-04-25 21:10:17.750180068 +0000 UTC m=+0.390441077 container attach 9140ba397d95bd7da9a1480436903107b4cd6daae3a5fd0e357b5e079e410ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:10:18.113 INFO:journalctl@ceph.osd.10.smithi097.stdout:Apr 25 21:10:17 smithi097 podman[70117]: 2024-04-25 21:10:17.968147629 +0000 UTC m=+0.608408639 container died 9140ba397d95bd7da9a1480436903107b4cd6daae3a5fd0e357b5e079e410ed5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:18.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.006+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:18.546 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.10.service' 2024-04-25T21:10:18.584 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:18.584 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-25T21:10:18.584 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-25T21:10:18.584 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.11 2024-04-25T21:10:18.872 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 systemd[1]: Stopping Ceph osd.11 for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:19.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.885+0000 7f4817780700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:19.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.885+0000 7f4817780700 -1 osd.11 106 *** Got signal Terminated *** 2024-04-25T21:10:19.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:18 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:18.885+0000 7f4817780700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-25T21:10:19.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:19.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:19.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:19.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:19.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:19.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:19.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:19 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:20.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:20 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:19.999+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:21.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:21 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:21.048+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:22.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:22 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:22.015+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6820 osd.8 since back 2024-04-25T21:09:58.906030+0000 front 2024-04-25T21:09:58.906072+0000 (oldest deadline 2024-04-25T21:10:21.706431+0000) 2024-04-25T21:10:23.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6806 osd.0 since back 2024-04-25T21:08:53.097251+0000 front 2024-04-25T21:08:53.097341+0000 (oldest deadline 2024-04-25T21:09:17.797226+0000) 2024-04-25T21:10:23.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2024-04-25T21:09:04.198575+0000 front 2024-04-25T21:09:04.198584+0000 (oldest deadline 2024-04-25T21:09:27.098618+0000) 2024-04-25T21:10:23.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6804 osd.2 since back 2024-04-25T21:09:09.999476+0000 front 2024-04-25T21:09:09.999455+0000 (oldest deadline 2024-04-25T21:09:34.600114+0000) 2024-04-25T21:10:23.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6814 osd.3 since back 2024-04-25T21:09:20.501262+0000 front 2024-04-25T21:09:20.501111+0000 (oldest deadline 2024-04-25T21:09:43.401088+0000) 2024-04-25T21:10:23.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2024-04-25T21:09:27.502029+0000 front 2024-04-25T21:09:27.502247+0000 (oldest deadline 2024-04-25T21:09:50.401893+0000) 2024-04-25T21:10:23.372 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6812 osd.5 since back 2024-04-25T21:09:34.403173+0000 front 2024-04-25T21:09:34.403108+0000 (oldest deadline 2024-04-25T21:09:57.303242+0000) 2024-04-25T21:10:23.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2024-04-25T21:09:52.505671+0000 front 2024-04-25T21:09:52.505552+0000 (oldest deadline 2024-04-25T21:10:14.805296+0000) 2024-04-25T21:10:23.373 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:23 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11[58266]: 2024-04-25T21:10:23.063+0000 7f480fd6c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.139:6820 osd.8 since back 2024-04-25T21:09:58.906030+0000 front 2024-04-25T21:09:58.906072+0000 (oldest deadline 2024-04-25T21:10:21.706431+0000) 2024-04-25T21:10:24.622 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:24 smithi139 podman[70870]: 2024-04-25 21:10:24.361778379 +0000 UTC m=+5.582618508 container died 0a69513a6b1dd5d7c18bd0abd76f5810957de038b77897d939ac175851ace4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:10:25.122 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:24 smithi139 podman[70870]: 2024-04-25 21:10:24.668931509 +0000 UTC m=+5.889771649 container cleanup 0a69513a6b1dd5d7c18bd0abd76f5810957de038b77897d939ac175851ace4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-25T21:10:25.122 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:24 smithi139 bash[70870]: ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11 2024-04-25T21:10:25.122 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:24 smithi139 podman[70887]: 2024-04-25 21:10:24.911730971 +0000 UTC m=+0.545884255 container remove 0a69513a6b1dd5d7c18bd0abd76f5810957de038b77897d939ac175851ace4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:10:25.497 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:25 smithi139 podman[70976]: 2024-04-25 21:10:25.143190899 +0000 UTC m=+0.017646719 image pull da4b004c09e820713483d039bc76fbd660be678f2ea3c03e059a6161b74dab76 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4 2024-04-25T21:10:25.498 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:25 smithi139 podman[70976]: 2024-04-25 21:10:25.254273728 +0000 UTC m=+0.128729519 container create f01bd76ef8d6c79833f4c7410aded4839f71ddc42a4af67782b5b6798ae0b64f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-25T21:10:25.793 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:25 smithi139 podman[70976]: 2024-04-25 21:10:25.498444372 +0000 UTC m=+0.372900175 container init f01bd76ef8d6c79833f4c7410aded4839f71ddc42a4af67782b5b6798ae0b64f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T21:10:25.793 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:25 smithi139 podman[70976]: 2024-04-25 21:10:25.502215759 +0000 UTC m=+0.376671553 container start f01bd76ef8d6c79833f4c7410aded4839f71ddc42a4af67782b5b6798ae0b64f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-25T21:10:25.793 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:25 smithi139 podman[70976]: 2024-04-25 21:10:25.59010784 +0000 UTC m=+0.464563645 container attach f01bd76ef8d6c79833f4c7410aded4839f71ddc42a4af67782b5b6798ae0b64f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, GIT_CLEAN=True) 2024-04-25T21:10:26.100 INFO:journalctl@ceph.osd.11.smithi139.stdout:Apr 25 21:10:25 smithi139 podman[70976]: 2024-04-25 21:10:25.794707611 +0000 UTC m=+0.669163410 container died f01bd76ef8d6c79833f4c7410aded4839f71ddc42a4af67782b5b6798ae0b64f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T21:10:26.445 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@osd.11.service' 2024-04-25T21:10:26.484 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:26.485 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-25T21:10:26.485 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-25T21:10:26.485 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.a 2024-04-25T21:10:26.781 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 21:10:26 smithi096 systemd[1]: Stopping Ceph mds.a for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:27.248 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 21:10:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-a[71074]: 2024-04-25T21:10:26.780+0000 7ff75b1b3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:27.249 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 21:10:26 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-a[71074]: 2024-04-25T21:10:26.780+0000 7ff75b1b3700 -1 mds.a *** got signal Terminated *** 2024-04-25T21:10:30.998 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 21:10:30 smithi096 podman[116865]: 2024-04-25 21:10:30.695865382 +0000 UTC m=+4.010853927 container died 49a4d99058f8e1b57f0f964c7b10c0924415aa5cc4e36a061789a98b7a3b3f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-a, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:10:31.439 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 21:10:31 smithi096 podman[116865]: 2024-04-25 21:10:31.238342543 +0000 UTC m=+4.553331105 container cleanup 49a4d99058f8e1b57f0f964c7b10c0924415aa5cc4e36a061789a98b7a3b3f53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-a, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:10:31.439 INFO:journalctl@ceph.mds.a.smithi096.stdout:Apr 25 21:10:31 smithi096 bash[116865]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-a 2024-04-25T21:10:31.651 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.a.service' 2024-04-25T21:10:31.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:31.685 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-25T21:10:31.685 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-25T21:10:31.685 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.d 2024-04-25T21:10:31.999 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 21:10:31 smithi096 systemd[1]: Stopping Ceph mds.d for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:31.999 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 21:10:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-d[69805]: 2024-04-25T21:10:31.991+0000 7fe471af6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:31.999 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 21:10:31 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-d[69805]: 2024-04-25T21:10:31.991+0000 7fe471af6700 -1 mds.d *** got signal Terminated *** 2024-04-25T21:10:33.783 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 21:10:33 smithi096 podman[116985]: 2024-04-25 21:10:33.528552244 +0000 UTC m=+1.627049861 container died 77a0792e48e7dd86f7cff845e110bf5d100d9ce898a59f0e072eafcf6a04c4d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-25T21:10:34.166 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 21:10:33 smithi096 podman[116985]: 2024-04-25 21:10:33.883122838 +0000 UTC m=+1.981620455 container cleanup 77a0792e48e7dd86f7cff845e110bf5d100d9ce898a59f0e072eafcf6a04c4d8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T21:10:34.167 INFO:journalctl@ceph.mds.d.smithi096.stdout:Apr 25 21:10:33 smithi096 bash[116985]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-d 2024-04-25T21:10:34.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.d.service' 2024-04-25T21:10:34.349 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:34.350 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-25T21:10:34.350 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-25T21:10:34.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.g 2024-04-25T21:10:34.678 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 21:10:34 smithi096 systemd[1]: Stopping Ceph mds.g for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:34.998 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 21:10:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g[70627]: 2024-04-25T21:10:34.678+0000 7fe42fd7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:34.998 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 21:10:34 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g[70627]: 2024-04-25T21:10:34.678+0000 7fe42fd7f700 -1 mds.g *** got signal Terminated *** 2024-04-25T21:10:38.927 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 21:10:38 smithi096 podman[117105]: 2024-04-25 21:10:38.624268675 +0000 UTC m=+4.062428013 container died 2984ac24830070b439e744497e79d18986af120acb5484207ec78a050f76cdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:10:38.927 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 21:10:38 smithi096 podman[117105]: 2024-04-25 21:10:38.927771036 +0000 UTC m=+4.365930363 container cleanup 2984ac24830070b439e744497e79d18986af120acb5484207ec78a050f76cdb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-25T21:10:39.178 INFO:journalctl@ceph.mds.g.smithi096.stdout:Apr 25 21:10:38 smithi096 bash[117105]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-g 2024-04-25T21:10:39.356 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.g.service' 2024-04-25T21:10:39.387 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:39.388 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-25T21:10:39.388 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-25T21:10:39.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.j 2024-04-25T21:10:39.697 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:39 smithi096 systemd[1]: Stopping Ceph mds.j for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:39.697 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-j[69342]: 2024-04-25T21:10:39.697+0000 7f33ef27b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:39.697 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:39 smithi096 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-j[69342]: 2024-04-25T21:10:39.697+0000 7f33ef27b700 -1 mds.j *** got signal Terminated *** 2024-04-25T21:10:41.430 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:41 smithi096 podman[117227]: 2024-04-25 21:10:41.121840017 +0000 UTC m=+1.520841281 container died 3ea429a3cc86be69a6bc8e18b0fe6db2b3de06f8d6c6a11f2706883c5b5944cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS) 2024-04-25T21:10:41.430 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:41 smithi096 podman[117227]: 2024-04-25 21:10:41.430991061 +0000 UTC m=+1.829992325 container cleanup 3ea429a3cc86be69a6bc8e18b0fe6db2b3de06f8d6c6a11f2706883c5b5944cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-25T21:10:41.748 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:41 smithi096 bash[117227]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-j 2024-04-25T21:10:41.749 INFO:journalctl@ceph.mds.j.smithi096.stdout:Apr 25 21:10:41 smithi096 podman[117241]: 2024-04-25 21:10:41.648664026 +0000 UTC m=+0.526749457 container remove 3ea429a3cc86be69a6bc8e18b0fe6db2b3de06f8d6c6a11f2706883c5b5944cb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-j, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-25T21:10:41.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.j.service' 2024-04-25T21:10:41.887 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:41.888 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-25T21:10:41.888 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-25T21:10:41.888 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.b 2024-04-25T21:10:42.195 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:10:41 smithi097 systemd[1]: Stopping Ceph mds.b for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:42.455 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:10:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: 2024-04-25T21:10:42.195+0000 7fdab1c29700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:42.455 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:10:42 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b[62908]: 2024-04-25T21:10:42.195+0000 7fdab1c29700 -1 mds.b *** got signal Terminated *** 2024-04-25T21:10:46.321 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:10:46 smithi097 podman[70222]: 2024-04-25 21:10:46.0345184 +0000 UTC m=+3.946823387 container died 37504cce1767de3997a9885a26682920bf853c47876783fa1d6e1d83d9b223c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-25T21:10:46.613 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:10:46 smithi097 podman[70222]: 2024-04-25 21:10:46.395313158 +0000 UTC m=+4.307618137 container cleanup 37504cce1767de3997a9885a26682920bf853c47876783fa1d6e1d83d9b223c1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-25T21:10:46.613 INFO:journalctl@ceph.mds.b.smithi097.stdout:Apr 25 21:10:46 smithi097 bash[70222]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-b 2024-04-25T21:10:46.821 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.b.service' 2024-04-25T21:10:46.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:46.855 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-25T21:10:46.856 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-25T21:10:46.856 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.e 2024-04-25T21:10:47.283 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:10:46 smithi097 systemd[1]: Stopping Ceph mds.e for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:47.612 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:10:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e[62282]: 2024-04-25T21:10:47.282+0000 7ff9de98f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:47.613 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:10:47 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e[62282]: 2024-04-25T21:10:47.282+0000 7ff9de98f700 -1 mds.e *** got signal Terminated *** 2024-04-25T21:10:50.362 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:10:50 smithi097 podman[70357]: 2024-04-25 21:10:50.041877129 +0000 UTC m=+2.967917040 container died 5fd78acc5bdd113464b133d4ba0e00e697fa6e1cea2ab68c0743e63cd23b716b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T21:10:50.682 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:10:50 smithi097 podman[70357]: 2024-04-25 21:10:50.47417722 +0000 UTC m=+3.400217173 container cleanup 5fd78acc5bdd113464b133d4ba0e00e697fa6e1cea2ab68c0743e63cd23b716b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:10:50.683 INFO:journalctl@ceph.mds.e.smithi097.stdout:Apr 25 21:10:50 smithi097 bash[70357]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-e 2024-04-25T21:10:50.898 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.e.service' 2024-04-25T21:10:50.932 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:50.932 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-25T21:10:50.932 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-25T21:10:50.932 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.h 2024-04-25T21:10:51.252 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:10:51 smithi097 systemd[1]: Stopping Ceph mds.h for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:51.612 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:10:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h[62600]: 2024-04-25T21:10:51.252+0000 7f1b6978d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:51.612 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:10:51 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h[62600]: 2024-04-25T21:10:51.252+0000 7f1b6978d700 -1 mds.h *** got signal Terminated *** 2024-04-25T21:10:54.335 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:10:54 smithi097 podman[70479]: 2024-04-25 21:10:54.048339012 +0000 UTC m=+2.907127416 container died f21fa8adbfc8ced7e7c593026aceedb239b96a91cd9cf45e89c5b33703a49d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-25T21:10:54.612 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:10:54 smithi097 podman[70479]: 2024-04-25 21:10:54.410621066 +0000 UTC m=+3.269409467 container cleanup f21fa8adbfc8ced7e7c593026aceedb239b96a91cd9cf45e89c5b33703a49d75 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-25T21:10:54.612 INFO:journalctl@ceph.mds.h.smithi097.stdout:Apr 25 21:10:54 smithi097 bash[70479]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-h 2024-04-25T21:10:54.777 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.h.service' 2024-04-25T21:10:54.808 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:54.808 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-25T21:10:54.808 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-25T21:10:54.808 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.k 2024-04-25T21:10:55.112 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:54 smithi097 systemd[1]: Stopping Ceph mds.k for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:55.612 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k[61970]: 2024-04-25T21:10:55.150+0000 7fbc2b7ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:55.612 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:55 smithi097 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k[61970]: 2024-04-25T21:10:55.150+0000 7fbc2b7ab700 -1 mds.k *** got signal Terminated *** 2024-04-25T21:10:57.006 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:56 smithi097 podman[70601]: 2024-04-25 21:10:56.683647265 +0000 UTC m=+1.648775284 container died d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2) 2024-04-25T21:10:57.290 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:57 smithi097 podman[70601]: 2024-04-25 21:10:57.106088267 +0000 UTC m=+2.071216279 container cleanup d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-d69f13d, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-25T21:10:57.290 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:57 smithi097 bash[70601]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k 2024-04-25T21:10:57.531 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.k.service' 2024-04-25T21:10:57.551 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:57 smithi097 podman[70615]: 2024-04-25 21:10:57.290402309 +0000 UTC m=+0.608779052 container remove d627f40e39c3f523cf0ba30290985415f756eefcdaf48498d787b5119fe39c05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-25T21:10:57.551 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:57 smithi097 systemd[1]: ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.k.service: Deactivated successfully. 2024-04-25T21:10:57.551 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:57 smithi097 systemd[1]: Stopped Ceph mds.k for b713778a-0344-11ef-bc93-c7b262605968. 2024-04-25T21:10:57.551 INFO:journalctl@ceph.mds.k.smithi097.stdout:Apr 25 21:10:57 smithi097 systemd[1]: ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.k.service: Consumed 1.074s CPU time. 2024-04-25T21:10:57.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:10:57.561 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-25T21:10:57.561 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-25T21:10:57.561 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.c 2024-04-25T21:10:57.872 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:10:57 smithi139 systemd[1]: Stopping Ceph mds.c for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:10:58.372 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:10:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: 2024-04-25T21:10:57.875+0000 7f99e6cf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:10:58.372 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:10:57 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c[63293]: 2024-04-25T21:10:57.875+0000 7f99e6cf6700 -1 mds.c *** got signal Terminated *** 2024-04-25T21:11:02.168 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:11:01 smithi139 podman[71082]: 2024-04-25 21:11:01.890603153 +0000 UTC m=+4.136500487 container died c1a2b34257bdca2dc2927c0ff96279b536b337b26f23eb15b013f38f946d953e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-25T21:11:02.517 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:11:02 smithi139 podman[71082]: 2024-04-25 21:11:02.275988193 +0000 UTC m=+4.521885532 container cleanup c1a2b34257bdca2dc2927c0ff96279b536b337b26f23eb15b013f38f946d953e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-d69f13d) 2024-04-25T21:11:02.518 INFO:journalctl@ceph.mds.c.smithi139.stdout:Apr 25 21:11:02 smithi139 bash[71082]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-c 2024-04-25T21:11:02.641 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.c.service' 2024-04-25T21:11:02.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:02.676 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-25T21:11:02.676 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-25T21:11:02.676 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.f 2024-04-25T21:11:02.871 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:11:02 smithi139 systemd[1]: Stopping Ceph mds.f for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:11:03.372 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:11:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f[62986]: 2024-04-25T21:11:02.995+0000 7ff7a1034700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:03.372 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:11:02 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f[62986]: 2024-04-25T21:11:02.995+0000 7ff7a1034700 -1 mds.f *** got signal Terminated *** 2024-04-25T21:11:06.320 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:11:06 smithi139 podman[71214]: 2024-04-25 21:11:06.041636089 +0000 UTC m=+3.141599148 container died c4c3a25e5f5c92edb64de85997da91280684eceab1e3b52e734f514450496cd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-25T21:11:06.622 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:11:06 smithi139 podman[71214]: 2024-04-25 21:11:06.4125865 +0000 UTC m=+3.512549566 container cleanup c4c3a25e5f5c92edb64de85997da91280684eceab1e3b52e734f514450496cd6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-25T21:11:06.622 INFO:journalctl@ceph.mds.f.smithi139.stdout:Apr 25 21:11:06 smithi139 bash[71214]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-f 2024-04-25T21:11:06.792 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.f.service' 2024-04-25T21:11:06.824 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:06.824 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-25T21:11:06.824 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-25T21:11:06.824 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.i 2024-04-25T21:11:07.122 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:11:06 smithi139 systemd[1]: Stopping Ceph mds.i for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:11:07.622 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:11:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i[63600]: 2024-04-25T21:11:07.165+0000 7f9e54962700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:07.622 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:11:07 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i[63600]: 2024-04-25T21:11:07.165+0000 7f9e54962700 -1 mds.i *** got signal Terminated *** 2024-04-25T21:11:10.333 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:11:10 smithi139 podman[71335]: 2024-04-25 21:11:10.043832313 +0000 UTC m=+2.998552410 container died d14a0dc08e86ede370eb50aaf72e15160c01bc938d2fb656cdc52bab94e8851b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-25T21:11:10.622 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:11:10 smithi139 podman[71335]: 2024-04-25 21:11:10.432809519 +0000 UTC m=+3.387529606 container cleanup d14a0dc08e86ede370eb50aaf72e15160c01bc938d2fb656cdc52bab94e8851b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-d69f13d, org.label-schema.build-date=20240416, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-25T21:11:10.622 INFO:journalctl@ceph.mds.i.smithi139.stdout:Apr 25 21:11:10 smithi139 bash[71335]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-i 2024-04-25T21:11:10.798 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.i.service' 2024-04-25T21:11:10.829 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:10.829 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-25T21:11:10.829 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-25T21:11:10.829 DEBUG:teuthology.orchestra.run.smithi139:> sudo systemctl stop ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.l 2024-04-25T21:11:11.122 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 21:11:10 smithi139 systemd[1]: Stopping Ceph mds.l for b713778a-0344-11ef-bc93-c7b262605968... 2024-04-25T21:11:11.621 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 21:11:11 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l[62671]: 2024-04-25T21:11:11.140+0000 7f25e5a93700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-25T21:11:11.622 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 21:11:11 smithi139 ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l[62671]: 2024-04-25T21:11:11.140+0000 7f25e5a93700 -1 mds.l *** got signal Terminated *** 2024-04-25T21:11:14.170 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 21:11:13 smithi139 podman[71455]: 2024-04-25 21:11:13.894818876 +0000 UTC m=+2.855287820 container died 226c852ad8e88ded88c1b9cc68f90828b3dc8f6434a364c54e40a6cd3b594453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l, RELEASE=wip-lusov-quiescer-d69f13d, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0) 2024-04-25T21:11:14.520 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 21:11:14 smithi139 podman[71455]: 2024-04-25 21:11:14.270110203 +0000 UTC m=+3.230579139 container cleanup 226c852ad8e88ded88c1b9cc68f90828b3dc8f6434a364c54e40a6cd3b594453 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:068a558e3af4932cf1c60dd321c0bdf500075eb52ea1bf455e0973ba114fbfd4, name=ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-d69f13d, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5) 2024-04-25T21:11:14.520 INFO:journalctl@ceph.mds.l.smithi139.stdout:Apr 25 21:11:14 smithi139 bash[71455]: ceph-b713778a-0344-11ef-bc93-c7b262605968-mds-l 2024-04-25T21:11:14.655 DEBUG:teuthology.orchestra.run.smithi139:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b713778a-0344-11ef-bc93-c7b262605968@mds.l.service' 2024-04-25T21:11:14.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-25T21:11:14.685 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-25T21:11:14.685 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b713778a-0344-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:11:14.875 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T21:11:17.288 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b713778a-0344-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:11:17.480 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T21:11:19.924 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b713778a-0344-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-25T21:11:20.111 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T21:11:22.410 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:11:22.442 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:11:22.473 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-25T21:11:22.498 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-25T21:11:22.499 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi096/crash 2024-04-25T21:11:22.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:11:22.536 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:11:22.536 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:11:22.538 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi097/crash 2024-04-25T21:11:22.539 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:11:22.577 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:11:22.577 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:11:22.579 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi139/crash 2024-04-25T21:11:22.580 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash -- . 2024-04-25T21:11:22.617 INFO:teuthology.orchestra.run.smithi139.stderr:tar: /var/lib/ceph/b713778a-0344-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-25T21:11:22.617 INFO:teuthology.orchestra.run.smithi139.stderr:tar: Error is not recoverable: exiting now 2024-04-25T21:11:22.618 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-25T21:11:22.619 DEBUG:teuthology.orchestra.run.smithi096:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-25T21:11:22.656 INFO:tasks.cephadm:Compressing logs... 2024-04-25T21:11:22.656 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:11:22.698 DEBUG:teuthology.orchestra.run.smithi097:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:11:22.701 DEBUG:teuthology.orchestra.run.smithi139:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T21:11:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:11:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:: No such file or directory 2024-04-25T21:11:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:11:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:11:22.722 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-25T21:11:22.723 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-25T21:11:22.723 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-25T21:11:22.723 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.g.log: /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-25T21:11:22.723 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.0.log: /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr: 87.4% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.a.log: 90.3% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr: 94.6% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr: 84.2% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-25T21:11:22.736 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:: No such file or directory 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:11:22.737 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.5% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.i.log: 90.5% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log: 83.9% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:11:22.738 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-25T21:11:22.740 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-25T21:11:22.741 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.11.log: 92.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-25T21:11:22.741 INFO:teuthology.orchestra.run.smithi139.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:11:22.744 INFO:teuthology.orchestra.run.smithi096.stderr: 91.1% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:: No such file or directory 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-25T21:11:22.745 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.7.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr: 87.5%/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.y.log: -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr: 90.4% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-25T21:11:22.746 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.e.log: 91.9% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-25T21:11:22.747 INFO:teuthology.orchestra.run.smithi097.stderr:gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-25T21:11:22.751 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-25T21:11:22.753 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-25T21:11:22.754 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:11:22.760 INFO:teuthology.orchestra.run.smithi097.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.k.log: 94.5% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-25T21:11:22.776 INFO:teuthology.orchestra.run.smithi139.stderr:/var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-25T21:11:22.777 INFO:teuthology.orchestra.run.smithi139.stderr: 91.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-25T21:11:22.797 INFO:teuthology.orchestra.run.smithi139.stderr: 94.2% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-25T21:11:23.042 INFO:teuthology.orchestra.run.smithi139.stderr: 93.9% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-25T21:11:23.101 INFO:teuthology.orchestra.run.smithi096.stderr: 90.2% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-25T21:11:23.177 INFO:teuthology.orchestra.run.smithi139.stderr: 91.9% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-25T21:11:23.188 INFO:teuthology.orchestra.run.smithi097.stderr: 92.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-25T21:11:23.272 INFO:teuthology.orchestra.run.smithi097.stderr: 94.5% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-25T21:11:23.958 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-25T21:11:24.320 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-25T21:11:24.713 INFO:teuthology.orchestra.run.smithi139.stderr: 96.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-25T21:11:24.912 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-25T21:11:25.996 INFO:teuthology.orchestra.run.smithi139.stderr: 94.3% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-25T21:11:26.290 INFO:teuthology.orchestra.run.smithi096.stderr: 93.9% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-25T21:11:26.355 INFO:teuthology.orchestra.run.smithi097.stderr: 94.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-25T21:11:26.458 INFO:teuthology.orchestra.run.smithi139.stderr: 94.0% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-25T21:11:26.946 INFO:teuthology.orchestra.run.smithi097.stderr: 93.8% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-25T21:11:28.268 INFO:teuthology.orchestra.run.smithi139.stderr: 94.2% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-25T21:11:28.370 INFO:teuthology.orchestra.run.smithi097.stderr: 94.1% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-25T21:11:28.385 INFO:teuthology.orchestra.run.smithi096.stderr: 93.5% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-25T21:11:28.770 INFO:teuthology.orchestra.run.smithi097.stderr: 94.2% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-25T21:11:28.786 INFO:teuthology.orchestra.run.smithi096.stderr: 93.8% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-25T21:11:28.788 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-04-25T21:11:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m6.076s 2024-04-25T21:11:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:user 0m17.846s 2024-04-25T21:11:28.788 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m0.676s 2024-04-25T21:11:30.154 INFO:teuthology.orchestra.run.smithi139.stderr: 94.1% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-25T21:11:30.156 INFO:teuthology.orchestra.run.smithi139.stderr: 2024-04-25T21:11:30.156 INFO:teuthology.orchestra.run.smithi139.stderr:real 0m7.441s 2024-04-25T21:11:30.156 INFO:teuthology.orchestra.run.smithi139.stderr:user 0m22.012s 2024-04-25T21:11:30.156 INFO:teuthology.orchestra.run.smithi139.stderr:sys 0m0.771s 2024-04-25T21:11:50.756 INFO:teuthology.orchestra.run.smithi097.stderr: 93.4% -- replaced with /var/log/ceph/b713778a-0344-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-25T21:11:50.758 INFO:teuthology.orchestra.run.smithi097.stderr: 2024-04-25T21:11:50.758 INFO:teuthology.orchestra.run.smithi097.stderr:real 0m28.042s 2024-04-25T21:11:50.758 INFO:teuthology.orchestra.run.smithi097.stderr:user 0m49.068s 2024-04-25T21:11:50.758 INFO:teuthology.orchestra.run.smithi097.stderr:sys 0m1.609s 2024-04-25T21:11:50.758 INFO:tasks.cephadm:Archiving logs... 2024-04-25T21:11:50.759 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi096/log 2024-04-25T21:11:50.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:11:53.327 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi097/log 2024-04-25T21:11:53.328 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:12:00.056 DEBUG:teuthology.misc:Transferring archived files from smithi139:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi139/log 2024-04-25T21:12:00.057 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T21:12:02.799 INFO:tasks.cephadm:Removing cluster... 2024-04-25T21:12:02.799 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b713778a-0344-11ef-bc93-c7b262605968 --force 2024-04-25T21:12:02.988 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T21:12:03.263 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b713778a-0344-11ef-bc93-c7b262605968 --force 2024-04-25T21:12:03.455 INFO:teuthology.orchestra.run.smithi097.stdout:Deleting cluster with fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T21:12:03.820 DEBUG:teuthology.orchestra.run.smithi139:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b713778a-0344-11ef-bc93-c7b262605968 --force 2024-04-25T21:12:04.011 INFO:teuthology.orchestra.run.smithi139.stdout:Deleting cluster with fsid: b713778a-0344-11ef-bc93-c7b262605968 2024-04-25T21:12:04.288 INFO:tasks.cephadm:Removing cephadm ... 2024-04-25T21:12:04.288 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:12:04.307 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:12:04.323 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-25T21:12:04.337 INFO:tasks.cephadm:Teardown complete 2024-04-25T21:12:04.338 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T21:12:04.350 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T21:12:04.350 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:12:04.353 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:12:04.365 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T21:12:04.474 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T21:12:04.491 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T21:12:04.491 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:12:04.494 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:12:04.496 DEBUG:teuthology.orchestra.run.smithi139:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T21:12:04.507 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:12:04.508 INFO:teuthology.orchestra.run.smithi097.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:12:04.510 INFO:teuthology.orchestra.run.smithi139.stderr:bash: line 1: ntpq: command not found 2024-04-25T21:12:04.511 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:12:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2024-04-25T21:12:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv01.front.sepia.ceph.com 3 6 377 23 +484us[ +487us] +/- 64ms 2024-04-25T21:12:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 22 -1177us[-1177us] +/- 70ms 2024-04-25T21:12:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 +722us[ +725us] +/- 78ms 2024-04-25T21:12:04.512 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:12:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:12:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-04-25T21:12:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 7 377 18 +475us[ +536us] +/- 64ms 2024-04-25T21:12:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 87 -1224us[-1163us] +/- 69ms 2024-04-25T21:12:04.513 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +682us[ +743us] +/- 78ms 2024-04-25T21:12:04.514 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:12:04.514 INFO:teuthology.orchestra.run.smithi139.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T21:12:04.515 INFO:teuthology.orchestra.run.smithi139.stdout:=============================================================================== 2024-04-25T21:12:04.515 INFO:teuthology.orchestra.run.smithi139.stdout:^* hv01.front.sepia.ceph.com 3 6 377 26 +509us[ +507us] +/- 64ms 2024-04-25T21:12:04.515 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 23 -1159us[-1159us] +/- 70ms 2024-04-25T21:12:04.515 INFO:teuthology.orchestra.run.smithi139.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +751us[ +751us] +/- 78ms 2024-04-25T21:12:04.515 INFO:teuthology.orchestra.run.smithi139.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T21:12:04.515 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T21:12:04.526 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T21:12:04.526 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T21:12:04.538 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:12:04.635 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:12:04.702 DEBUG:teuthology.orchestra.run.smithi139:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T21:12:04.766 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:12:04.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:12:04.795 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:12:04.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:12:04.822 DEBUG:teuthology.orchestra.run.smithi139:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-25T21:12:04.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:12:04.849 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T21:12:04.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T21:12:04.870 INFO:teuthology.task.internal:Duration was 1934.825801 seconds 2024-04-25T21:12:04.871 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T21:12:04.881 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T21:12:04.881 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:12:04.884 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:12:04.886 DEBUG:teuthology.orchestra.run.smithi139:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T21:12:04.919 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:12:04.921 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:12:04.924 INFO:teuthology.orchestra.run.smithi139.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T21:12:05.403 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T21:12:05.403 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-04-25T21:12:05.404 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:12:05.425 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2024-04-25T21:12:05.425 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:12:05.470 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi139.front.sepia.ceph.com 2024-04-25T21:12:05.470 DEBUG:teuthology.orchestra.run.smithi139:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T21:12:05.504 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T21:12:05.504 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:12:05.507 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:12:05.512 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T21:12:05.569 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T21:12:05.569 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:12:05.574 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:12:05.578 DEBUG:teuthology.orchestra.run.smithi139:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T21:12:06.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T21:12:06.185 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T21:12:06.185 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:12:06.208 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:12:06.232 DEBUG:teuthology.orchestra.run.smithi139:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T21:12:06.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T21:12:06.268 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:12:06.271 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:12:06.275 DEBUG:teuthology.orchestra.run.smithi139:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:12:06.290 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-04-25T21:12:06.296 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2024-04-25T21:12:06.319 INFO:teuthology.orchestra.run.smithi139.stdout:kernel.core_pattern = core 2024-04-25T21:12:06.337 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:12:06.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:12:06.362 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:12:06.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:12:06.379 DEBUG:teuthology.orchestra.run.smithi139:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T21:12:06.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T21:12:06.395 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T21:12:06.407 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T21:12:06.408 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi096 2024-04-25T21:12:06.408 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:12:06.455 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi097 2024-04-25T21:12:06.455 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:12:06.490 DEBUG:teuthology.misc:Transferring archived files from smithi139:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652/remote/smithi139 2024-04-25T21:12:06.490 DEBUG:teuthology.orchestra.run.smithi139:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T21:12:06.526 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T21:12:06.527 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:12:06.529 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:12:06.531 DEBUG:teuthology.orchestra.run.smithi139:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T21:12:06.574 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T21:12:06.588 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T21:12:06.588 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T21:12:06.600 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T21:12:06.600 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:12:06.603 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:12:06.605 DEBUG:teuthology.orchestra.run.smithi139:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T21:12:06.614 INFO:teuthology.orchestra.run.smithi096.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:12 /home/ubuntu/cephtest 2024-04-25T21:12:06.616 INFO:teuthology.orchestra.run.smithi097.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:12 /home/ubuntu/cephtest 2024-04-25T21:12:06.629 INFO:teuthology.orchestra.run.smithi139.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 21:12 /home/ubuntu/cephtest 2024-04-25T21:12:06.630 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T21:12:06.642 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T21:12:06.668 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T21:12:06.696 DEBUG:teuthology.nuke:shortname: smithi096 2024-04-25T21:12:06.696 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T21:12:06.719 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:34:11.728484', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIcMo5SVjzeCCC9sX+g6yCA3BHi6bGDMpaeg4N3zUpJNBk/2sx1ywEcH0R/R6D/f+T/VpBpD87A1Sk+bVp43+h0='} 2024-04-25T21:12:06.725 DEBUG:teuthology.nuke:shortname: smithi097 2024-04-25T21:12:06.725 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T21:12:06.746 DEBUG:teuthology.nuke:shortname: smithi139 2024-04-25T21:12:06.746 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T21:12:06.749 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:34:11.730323', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI10lUouKjO775cv66l7qvkSNBu26z/d1nD6syQgF66fa5DXV/dvpIqnAE1ec6W62FPkrM3dkp2j2LhOAYNI9uk='} 2024-04-25T21:12:06.762 INFO:teuthology.orchestra.console.smithi096:Power off 2024-04-25T21:12:06.762 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T21:12:06.773 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi139.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-25_19:59:35-fs-wip-lusov-quiescer-distro-default-smithi/7673652', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 20:34:11.731662', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDQ507FFNV3QplQfR3lBTH6lwv6uEKPVXzfIlGTtaFkZBYIEzDHzi1y+Il8x28vMzI9kkN+N5K+4HkmkLXLIGAE='} 2024-04-25T21:12:06.786 DEBUG:teuthology.orchestra.console.smithi096:power off output: Chassis Power Control: Down/Off 2024-04-25T21:12:06.786 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:06.802 INFO:teuthology.orchestra.console.smithi097:Power off 2024-04-25T21:12:06.803 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T21:12:06.815 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-04-25T21:12:06.826 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2024-04-25T21:12:06.826 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:06.841 INFO:teuthology.orchestra.console.smithi139:Power off 2024-04-25T21:12:06.841 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T21:12:06.849 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-25T21:12:06.862 DEBUG:teuthology.orchestra.console.smithi139:power off output: Chassis Power Control: Down/Off 2024-04-25T21:12:06.862 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:06.888 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2024-04-25T21:12:10.816 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:10.851 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:10.890 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:10.926 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is on 2024-04-25T21:12:10.959 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-25T21:12:10.999 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is on 2024-04-25T21:12:14.930 DEBUG:teuthology.orchestra.console.smithi096:pexpect command: ipmitool -H smithi096.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:14.960 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:15.000 DEBUG:teuthology.orchestra.console.smithi139:pexpect command: ipmitool -H smithi139.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:15.041 DEBUG:teuthology.orchestra.console.smithi096:check power output: Chassis Power is off 2024-04-25T21:12:15.069 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2024-04-25T21:12:15.109 DEBUG:teuthology.orchestra.console.smithi139:check power output: Chassis Power is off 2024-04-25T21:12:15.142 INFO:teuthology.orchestra.console.smithi096:Power off completed 2024-04-25T21:12:15.210 INFO:teuthology.orchestra.console.smithi139:Power off completed 2024-04-25T21:12:19.070 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T21:12:19.180 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2024-04-25T21:12:19.281 INFO:teuthology.orchestra.console.smithi097:Power off completed 2024-04-25T21:12:19.385 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/no} objectstore-ec/bluestore-bitmap omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 1934.8258006572723 failure_reason: 'error during quiesce thrashing: Error quiescing set ''2c77974d'': 110 (ETIMEDOUT)' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-25T21:12:19.385 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T21:12:19.474 INFO:teuthology.run:FAIL